[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:38:42 dist-test-slave-p3t5 krb5kdc[20040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:38:42 dist-test-slave-p3t5 krb5kdc[20040](info): set up 2 sockets
Aug 08 14:38:42 dist-test-slave-p3t5 krb5kdc[20040](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 14:38:43.448527 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.008s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:38:43 dist-test-slave-p3t5 krb5kdc[20040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:38:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:38:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:38:43.594837 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:33595
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:33595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:43.715329 20053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:43.715629 20053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:43.715672 20053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:43.719761 20053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:38:43.719832 20053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:43.719852 20053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:43.719870 20053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:38:43.719892 20053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:38:43.725425 20053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:33595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:33595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:43.726577 20053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:43.727713 20053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:43.737241 20058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:43.737242 20059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:43.737246 20061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:43.737533 20053 server_base.cc:1034] running on GCE node
I20260808 14:38:43.738224 20053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:43.739512 20053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:43.740725 20053 hybrid_clock.cc:648] HybridClock initialized: now 1786199923740668 us; error 71 us; skew 500 ppm
Aug 08 14:38:43 dist-test-slave-p3t5 krb5kdc[20040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199923, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:43.743950 20053 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:38:43.745050 20053 webserver.cc:460] Webserver started at http://127.19.144.126:44473/ using document root <none> and password file <none>
I20260808 14:38:43.745870 20053 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:43.745973 20053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:43.746282 20053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:43.748612 20053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "c92a8ce0723a48ecadb99133de623096"
format_stamp: "Formatted at 2026-08-08 14:38:43 on dist-test-slave-p3t5"
server_key: "e1b71370bb25138c53fa6b42591e2c8b"
server_key_iv: "5a49f8a5f1077ad6932ef81db05f7086"
server_key_version: "encryptionkey@0"
I20260808 14:38:43.749189 20053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "c92a8ce0723a48ecadb99133de623096"
format_stamp: "Formatted at 2026-08-08 14:38:43 on dist-test-slave-p3t5"
server_key: "e1b71370bb25138c53fa6b42591e2c8b"
server_key_iv: "5a49f8a5f1077ad6932ef81db05f7086"
server_key_version: "encryptionkey@0"
I20260808 14:38:43.753432 20053 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:38:43.756263 20068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:43.757926 20053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:38:43.758108 20053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "c92a8ce0723a48ecadb99133de623096"
format_stamp: "Formatted at 2026-08-08 14:38:43 on dist-test-slave-p3t5"
server_key: "e1b71370bb25138c53fa6b42591e2c8b"
server_key_iv: "5a49f8a5f1077ad6932ef81db05f7086"
server_key_version: "encryptionkey@0"
I20260808 14:38:43.758260 20053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:43.778574 20053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:43.782419 20053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:43.782676 20053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:43.791134 20053 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:33595
I20260808 14:38:43.791157 20120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:33595 every 8 connection(s)
I20260808 14:38:43.792244 20053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:38:43.794185 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 20053
I20260808 14:38:43.794358 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:38:43.795524 20121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:43.795828 20033 external_mini_cluster.cc:1428] Setting key cb9d395a910f39a679d04168733406a1
I20260808 14:38:43.802573 20121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096: Bootstrap starting.
I20260808 14:38:43.805671 20121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:43.806632 20121 log.cc:824] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096: Log is configured to *not* fsync() on all Append() calls
I20260808 14:38:43.808951 20121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096: No bootstrap required, opened a new log
I20260808 14:38:43.812686 20121 raft_consensus.cc:348] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92a8ce0723a48ecadb99133de623096" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33595 } }
I20260808 14:38:43.812933 20121 raft_consensus.cc:374] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:43.812969 20121 raft_consensus.cc:729] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c92a8ce0723a48ecadb99133de623096, State: Initialized, Role: FOLLOWER
I20260808 14:38:43.813539 20121 consensus_queue.cc:260] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92a8ce0723a48ecadb99133de623096" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33595 } }
I20260808 14:38:43.813757 20121 raft_consensus.cc:388] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:38:43.813858 20121 raft_consensus.cc:482] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:38:43.814002 20121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:43.814987 20121 raft_consensus.cc:504] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92a8ce0723a48ecadb99133de623096" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33595 } }
I20260808 14:38:43.815414 20121 leader_election.cc:302] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c92a8ce0723a48ecadb99133de623096; no voters:
I20260808 14:38:43.815825 20121 leader_election.cc:288] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:38:43.815936 20127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [term 1 FOLLOWER]: Leader election won for term 1
Aug 08 14:38:43 dist-test-slave-p3t5 krb5kdc[20040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:43.816221 20127 raft_consensus.cc:686] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [term 1 LEADER]: Becoming Leader. State: Replica: c92a8ce0723a48ecadb99133de623096, State: Running, Role: LEADER
I20260808 14:38:43.816612 20127 consensus_queue.cc:237] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92a8ce0723a48ecadb99133de623096" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33595 } }
I20260808 14:38:43.817332 20121 sys_catalog.cc:564] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:38:43.818737 20128 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c92a8ce0723a48ecadb99133de623096" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92a8ce0723a48ecadb99133de623096" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33595 } } }
I20260808 14:38:43.818787 20129 sys_catalog.cc:455] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c92a8ce0723a48ecadb99133de623096. Latest consensus state: current_term: 1 leader_uuid: "c92a8ce0723a48ecadb99133de623096" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c92a8ce0723a48ecadb99133de623096" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33595 } } }
I20260808 14:38:43.818895 20128 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [sys.catalog]: This master's current role is: LEADER
I20260808 14:38:43.818902 20129 sys_catalog.cc:458] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096 [sys.catalog]: This master's current role is: LEADER
I20260808 14:38:43.819401 20136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:38:43.823077 20136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:38:43.824012 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:43.799414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44464 (local address 127.19.144.126:33595)
0808 14:38:43.799983 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:43.800004 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:43.801081 (+ 1077us) server_negotiation.cc:407] Connection header received
0808 14:38:43.805699 (+ 4618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:43.805725 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:43.806184 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:43.806571 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:43.809192 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:43.810311 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:43.811582 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:43.812073 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:43.812270 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:43.817113 (+ 4843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:43.817121 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:43.817138 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:43.817177 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:43.819814 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:43.820521 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:43.820534 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:43.820540 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:43.820683 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:43.821106 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:43.821111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:43.821115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:43.821783 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:43.822009 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:43.822663 (+ 654us) server_negotiation.cc:299] Negotiation successful
0808 14:38:43.822966 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":175,"threads_started":1}
I20260808 14:38:43.831768 20136 catalog_manager.cc:1347] Generated new cluster ID: 50c7b2642deb42ebaabea03ba40445b6
I20260808 14:38:43.831856 20136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:38:43.839694 20136 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:38:43.840725 20136 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:38:43.846794 20136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c92a8ce0723a48ecadb99133de623096: Generated new TSK 0
I20260808 14:38:43.847679 20136 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:38:43.894323 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:33595
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:44.025440 20148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:44.025763 20148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:44.025810 20148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:44.029974 20148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:44.030051 20148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:44.030115 20148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:38:44.035663 20148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:33595
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:44.036866 20148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:44.038005 20148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:44.048566 20154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:44.048566 20156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:44.049060 20153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:44.049160 20148 server_base.cc:1034] running on GCE node
I20260808 14:38:44.049604 20148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:44.050398 20148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:44.051576 20148 hybrid_clock.cc:648] HybridClock initialized: now 1786199924051557 us; error 33 us; skew 500 ppm
Aug 08 14:38:44 dist-test-slave-p3t5 krb5kdc[20040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199924, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:44.054749 20148 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:38:44.055794 20148 webserver.cc:460] Webserver started at http://127.19.144.65:42601/ using document root <none> and password file <none>
I20260808 14:38:44.056421 20148 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:44.056489 20148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:44.056726 20148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:44.059059 20148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "738ebdef6a9e41518e86c98f3d2874b7"
format_stamp: "Formatted at 2026-08-08 14:38:44 on dist-test-slave-p3t5"
server_key: "36ed2721a275c630e9233c23417ad80c"
server_key_iv: "1ce11f53f4a7ada8d79ba77001cedef0"
server_key_version: "encryptionkey@0"
I20260808 14:38:44.059684 20148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "738ebdef6a9e41518e86c98f3d2874b7"
format_stamp: "Formatted at 2026-08-08 14:38:44 on dist-test-slave-p3t5"
server_key: "36ed2721a275c630e9233c23417ad80c"
server_key_iv: "1ce11f53f4a7ada8d79ba77001cedef0"
server_key_version: "encryptionkey@0"
I20260808 14:38:44.064142 20148 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:38:44.067060 20163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:44.068557 20148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 14:38:44.068710 20148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "738ebdef6a9e41518e86c98f3d2874b7"
format_stamp: "Formatted at 2026-08-08 14:38:44 on dist-test-slave-p3t5"
server_key: "36ed2721a275c630e9233c23417ad80c"
server_key_iv: "1ce11f53f4a7ada8d79ba77001cedef0"
server_key_version: "encryptionkey@0"
I20260808 14:38:44.068841 20148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:44.093066 20148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:44.096873 20148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:44.097112 20148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:44.097824 20148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:38:44.098753 20148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:38:44.098806 20148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:44.098882 20148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:38:44.098914 20148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:44.108536 20148 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:34711
I20260808 14:38:44.108561 20277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:34711 every 8 connection(s)
I20260808 14:38:44.109534 20148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:38:44.114269 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 20148
I20260808 14:38:44.114395 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:38:44.114684 20033 external_mini_cluster.cc:1428] Setting key 1cc70d0b885fec1ac30916096b50f226
Aug 08 14:38:44 dist-test-slave-p3t5 krb5kdc[20040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199924, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:44.126125 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:44.111695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:42487 (local address 127.19.144.126:33595)
0808 14:38:44.111899 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:44.111905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:44.113116 (+ 1211us) server_negotiation.cc:407] Connection header received
0808 14:38:44.114103 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:44.114108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:44.114177 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:44.114314 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:44.115713 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:44.116248 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:44.117532 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:44.117861 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:44.117986 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:44.121346 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:44.121353 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:44.121360 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:44.121407 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:44.123530 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:44.124198 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:44.124202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:44.124204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:44.124310 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:44.124697 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:44.124701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:44.124703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:44.124979 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:44.125095 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:44.125741 (+ 646us) server_negotiation.cc:299] Negotiation successful
0808 14:38:44.125899 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:38:44.127211 20280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:44.112222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:33595 (local address 127.19.144.65:42487)
0808 14:38:44.112939 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:44.112983 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:44.113835 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:44.114553 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:44.114561 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:44.115021 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:44.115529 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:44.115543 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:44.116383 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:44.116386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:44.117398 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:44.117409 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:44.118031 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:44.118036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:44.118278 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:44.119035 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:44.119062 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:44.121134 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:44.123683 (+ 2549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:44.123690 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:44.123707 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:44.124068 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:44.124417 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:44.124419 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:44.124421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:44.124591 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:44.125102 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:44.125109 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:44.125408 (+ 299us) client_negotiation.cc:769] Sending connection context
0808 14:38:44.125594 (+ 186us) client_negotiation.cc:240] Negotiation successful
0808 14:38:44.126048 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":140,"threads_started":1}
I20260808 14:38:44.128842 20278 heartbeater.cc:347] Connected to a master server at 127.19.144.126:33595
I20260808 14:38:44.129277 20278 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
I20260808 14:38:44.130084 20278 heartbeater.cc:511] Master 127.19.144.126:33595 requested a full tablet report, sending...
I20260808 14:38:44.132242 20085 ts_manager.cc:194] Registered new tserver with Master: 738ebdef6a9e41518e86c98f3d2874b7 (127.19.144.65:34711)
I20260808 14:38:44.133574 20085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:42487
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:38:44.150033 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:33595
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:44.271555 20283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:44.271919 20283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:44.272008 20283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:44.276353 20283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:44.276467 20283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:44.276566 20283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:38:44.282200 20283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:33595
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:44.283473 20283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:44.284649 20283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:44.296420 20291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:44.296445 20289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:44.296448 20288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:44.296685 20283 server_base.cc:1034] running on GCE node
I20260808 14:38:44.297308 20283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:44.298193 20283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:44.299500 20283 hybrid_clock.cc:648] HybridClock initialized: now 1786199924299426 us; error 29 us; skew 500 ppm
Aug 08 14:38:44 dist-test-slave-p3t5 krb5kdc[20040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199924, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:44.302735 20283 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:38:44.303741 20283 webserver.cc:460] Webserver started at http://127.19.144.66:39135/ using document root <none> and password file <none>
I20260808 14:38:44.304395 20283 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:44.304502 20283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:44.304816 20283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:44.307453 20283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "cb45a3c5da4544a1ae0eadf107b63a24"
format_stamp: "Formatted at 2026-08-08 14:38:44 on dist-test-slave-p3t5"
server_key: "0e8760089fbcfef4b29d1353deb3b713"
server_key_iv: "c03695423889a2f27616307a89db5675"
server_key_version: "encryptionkey@0"
I20260808 14:38:44.308144 20283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "cb45a3c5da4544a1ae0eadf107b63a24"
format_stamp: "Formatted at 2026-08-08 14:38:44 on dist-test-slave-p3t5"
server_key: "0e8760089fbcfef4b29d1353deb3b713"
server_key_iv: "c03695423889a2f27616307a89db5675"
server_key_version: "encryptionkey@0"
I20260808 14:38:44.312338 20283 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 14:38:44.315129 20298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:44.316542 20283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 14:38:44.316697 20283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "cb45a3c5da4544a1ae0eadf107b63a24"
format_stamp: "Formatted at 2026-08-08 14:38:44 on dist-test-slave-p3t5"
server_key: "0e8760089fbcfef4b29d1353deb3b713"
server_key_iv: "c03695423889a2f27616307a89db5675"
server_key_version: "encryptionkey@0"
I20260808 14:38:44.316833 20283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:44.334172 20283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:44.337906 20283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:44.338135 20283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:44.338824 20283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:38:44.339756 20283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:38:44.339807 20283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:44.339882 20283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:38:44.339918 20283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:44.349684 20283 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:38319
I20260808 14:38:44.349759 20414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:38319 every 8 connection(s)
I20260808 14:38:44.350704 20283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:38:44.358604 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 20283
I20260808 14:38:44.358711 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:38:44.358985 20033 external_mini_cluster.cc:1428] Setting key 24ad4a22b596d4de98b73979f4999d39
Aug 08 14:38:44 dist-test-slave-p3t5 krb5kdc[20040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199924, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:44.366569 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:44.352677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:52101 (local address 127.19.144.126:33595)
0808 14:38:44.352833 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:44.352838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:44.354304 (+ 1466us) server_negotiation.cc:407] Connection header received
0808 14:38:44.355296 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:44.355300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:44.355371 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:44.355488 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:44.356822 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:44.357377 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:44.358649 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:44.358938 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:44.359061 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:44.362094 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:44.362098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:44.362100 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:44.362126 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:44.363924 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:44.364634 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:44.364638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:44.364639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:44.364730 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:44.365170 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:44.365172 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:44.365173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:44.365389 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:44.365469 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:44.366314 (+ 845us) server_negotiation.cc:299] Negotiation successful
0808 14:38:44.366439 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:38:44.368060 20417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:44.353031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:33595 (local address 127.19.144.66:52101)
0808 14:38:44.354113 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:44.354163 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:44.355067 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:44.355687 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:44.355695 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:44.356145 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:44.356661 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:44.356676 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:44.357519 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:44.357523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:44.358509 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:44.358519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:44.359112 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:44.359117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:44.359299 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:44.359972 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:44.359990 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:44.361908 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:44.364057 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:44.364064 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:44.364083 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:44.364503 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:44.364832 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:44.364838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:44.364843 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:44.365063 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:44.365503 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:44.365514 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:44.366007 (+ 493us) client_negotiation.cc:769] Sending connection context
0808 14:38:44.366258 (+ 251us) client_negotiation.cc:240] Negotiation successful
0808 14:38:44.366693 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"thread_start_us":649,"threads_started":1}
I20260808 14:38:44.369469 20415 heartbeater.cc:347] Connected to a master server at 127.19.144.126:33595
I20260808 14:38:44.369800 20415 heartbeater.cc:464] Registering TS with master...
I20260808 14:38:44.370394 20415 heartbeater.cc:511] Master 127.19.144.126:33595 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
I20260808 14:38:44.371594 20085 ts_manager.cc:194] Registered new tserver with Master: cb45a3c5da4544a1ae0eadf107b63a24 (127.19.144.66:38319)
I20260808 14:38:44.372287 20085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:52101
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:38:44.394027 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:33595
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:44.514917 20420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:44.515249 20420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:44.515336 20420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:44.519730 20420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:44.519879 20420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:44.519974 20420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:38:44.525528 20420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:33595
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:44.526690 20420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:44.527803 20420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:38:44.537983 20420 server_base.cc:1034] running on GCE node
W20260808 14:38:44.538004 20428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:44.537966 20425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:44.537993 20426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:44.538757 20420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:44.539451 20420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:44.540664 20420 hybrid_clock.cc:648] HybridClock initialized: now 1786199924540641 us; error 34 us; skew 500 ppm
Aug 08 14:38:44 dist-test-slave-p3t5 krb5kdc[20040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199924, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:44.543763 20420 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:38:44.544847 20420 webserver.cc:460] Webserver started at http://127.19.144.67:32939/ using document root <none> and password file <none>
I20260808 14:38:44.545498 20420 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:44.545603 20420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:44.545888 20420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:44.548368 20420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "5bea84c5b73a49b3a9b874e23875ecaf"
format_stamp: "Formatted at 2026-08-08 14:38:44 on dist-test-slave-p3t5"
server_key: "57601d6cf11db8039c2f6d24ee9ee3de"
server_key_iv: "886d5c8d8b14df077e8851bf49809f86"
server_key_version: "encryptionkey@0"
I20260808 14:38:44.548975 20420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "5bea84c5b73a49b3a9b874e23875ecaf"
format_stamp: "Formatted at 2026-08-08 14:38:44 on dist-test-slave-p3t5"
server_key: "57601d6cf11db8039c2f6d24ee9ee3de"
server_key_iv: "886d5c8d8b14df077e8851bf49809f86"
server_key_version: "encryptionkey@0"
I20260808 14:38:44.553200 20420 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 14:38:44.555979 20435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:44.557385 20420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:38:44.557562 20420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "5bea84c5b73a49b3a9b874e23875ecaf"
format_stamp: "Formatted at 2026-08-08 14:38:44 on dist-test-slave-p3t5"
server_key: "57601d6cf11db8039c2f6d24ee9ee3de"
server_key_iv: "886d5c8d8b14df077e8851bf49809f86"
server_key_version: "encryptionkey@0"
I20260808 14:38:44.557731 20420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:44.581106 20420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:44.584882 20420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:44.585115 20420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:44.585860 20420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:38:44.586790 20420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:38:44.586846 20420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:44.586918 20420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:38:44.586957 20420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:44.596753 20420 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:41619
I20260808 14:38:44.596772 20550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:41619 every 8 connection(s)
I20260808 14:38:44.597838 20420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:38:44.603363 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 20420
I20260808 14:38:44.603525 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:38:44.603855 20033 external_mini_cluster.cc:1428] Setting key 7d4a3746db379229b605470ec4b4c9f4
Aug 08 14:38:44 dist-test-slave-p3t5 krb5kdc[20040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199924, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:44.612576 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:44.599812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:40693 (local address 127.19.144.126:33595)
0808 14:38:44.600013 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:44.600017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:44.601156 (+ 1139us) server_negotiation.cc:407] Connection header received
0808 14:38:44.602194 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:44.602197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:44.602248 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:44.602331 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:44.603704 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:44.604497 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:44.606090 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:44.606320 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:44.606441 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:44.609018 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:44.609029 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:44.609032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:44.609056 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:44.610401 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:44.610954 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:44.610962 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:44.610963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:44.611043 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:44.611446 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:44.611449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:44.611451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:44.611647 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:44.611756 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:44.612280 (+ 524us) server_negotiation.cc:299] Negotiation successful
0808 14:38:44.612409 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 14:38:44.613454 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:44.600255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:33595 (local address 127.19.144.67:40693)
0808 14:38:44.600992 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:44.601039 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:44.601987 (+ 948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:44.602535 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:44.602542 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:44.602969 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:44.603517 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:44.603543 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:44.604643 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:44.604647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:44.605951 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:44.605963 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:44.606488 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:44.606492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:44.606703 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:44.607339 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:44.607355 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:44.608846 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:44.610535 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:44.610541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:44.610557 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:44.610832 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:44.611146 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:44.611149 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:44.611151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:44.611324 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:44.611767 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:44.611775 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:44.612036 (+ 261us) client_negotiation.cc:769] Sending connection context
0808 14:38:44.612238 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 14:38:44.612594 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":123,"threads_started":1}
I20260808 14:38:44.614635 20551 heartbeater.cc:347] Connected to a master server at 127.19.144.126:33595
I20260808 14:38:44.614912 20551 heartbeater.cc:464] Registering TS with master...
I20260808 14:38:44.615451 20551 heartbeater.cc:511] Master 127.19.144.126:33595 requested a full tablet report, sending...
I20260808 14:38:44.616559 20085 ts_manager.cc:194] Registered new tserver with Master: 5bea84c5b73a49b3a9b874e23875ecaf (127.19.144.67:41619)
I20260808 14:38:44.617175 20085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:40693
I20260808 14:38:44.618875 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 14:38:44 dist-test-slave-p3t5 krb5kdc[20040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199924, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 14:38:44 dist-test-slave-p3t5 krb5kdc[20040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199924, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.144.65@KRBTEST.COM
I20260808 14:38:44.648013 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:44.638071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45448 (local address 127.19.144.65:34711)
0808 14:38:44.638337 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:44.638346 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:44.638384 (+ 38us) server_negotiation.cc:407] Connection header received
0808 14:38:44.638440 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:44.638445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:44.638660 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:44.638800 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:44.639531 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:44.640338 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:44.641160 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:44.641542 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:44.641685 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:44.644306 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:44.644317 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:44.644333 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:44.644361 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:44.646193 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:44.646671 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:44.646677 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:44.646678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:44.646789 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:44.647084 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:44.647086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:44.647088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:44.647361 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:44.647554 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:44.647696 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 14:38:44.647863 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":86,"threads_started":1}
W20260808 14:38:44.649377 20192 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45448
Aug 08 14:38:44 dist-test-slave-p3t5 krb5kdc[20040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 14:38:44 dist-test-slave-p3t5 krb5kdc[20040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.65@KRBTEST.COM
I20260808 14:38:44.679311 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:44.670023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45454 (local address 127.19.144.65:34711)
0808 14:38:44.670171 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:44.670178 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:44.670285 (+ 107us) server_negotiation.cc:407] Connection header received
0808 14:38:44.670477 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:44.670481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:44.670529 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:44.670619 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:44.671417 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:44.671948 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:44.672948 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:44.673322 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:44.673433 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:44.676069 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:44.676080 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:44.676083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:44.676109 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:44.677428 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:44.677981 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:44.677985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:44.677987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:44.678079 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:44.678371 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:44.678375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:44.678376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:44.678568 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:44.678719 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:44.678987 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 14:38:44.679124 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 14:38:44.680576 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 20148
I20260808 14:38:44.688236 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 20283
I20260808 14:38:44.695035 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 20420
I20260808 14:38:44.703091 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 20053
2026-08-08T14:38:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2347 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:38:44 dist-test-slave-p3t5 krb5kdc[20565](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:38:44 dist-test-slave-p3t5 krb5kdc[20565](info): set up 2 sockets
Aug 08 14:38:44 dist-test-slave-p3t5 krb5kdc[20565](info): commencing operation
krb5kdc: starting...
W20260808 14:38:45.775090 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:38:45 dist-test-slave-p3t5 krb5kdc[20565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:38:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:38:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:38:45.913051 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:39787
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:39787
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:46.037002 20578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:46.037366 20578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:46.037446 20578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:46.041778 20578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:38:46.041877 20578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:46.041913 20578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:46.041954 20578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:38:46.041990 20578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:38:46.047750 20578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:39787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:39787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:46.049117 20578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:46.050431 20578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:46.060070 20583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:46.060108 20586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:46.060108 20584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:46.060403 20578 server_base.cc:1034] running on GCE node
I20260808 14:38:46.061040 20578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:46.062410 20578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:46.063604 20578 hybrid_clock.cc:648] HybridClock initialized: now 1786199926063563 us; error 45 us; skew 500 ppm
Aug 08 14:38:46 dist-test-slave-p3t5 krb5kdc[20565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199926, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:46.071456 20578 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:38:46.072580 20578 webserver.cc:460] Webserver started at http://127.19.144.126:46491/ using document root <none> and password file <none>
I20260808 14:38:46.073186 20578 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:46.073249 20578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:46.073434 20578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:46.075798 20578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "895cfff4c0bd4e27884296899b584d5d"
format_stamp: "Formatted at 2026-08-08 14:38:46 on dist-test-slave-p3t5"
server_key: "5767271c81dd9715eb33cee2477d8fd6"
server_key_iv: "c071bd62ed91fede2d046c6e86d692f9"
server_key_version: "encryptionkey@0"
I20260808 14:38:46.076360 20578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "895cfff4c0bd4e27884296899b584d5d"
format_stamp: "Formatted at 2026-08-08 14:38:46 on dist-test-slave-p3t5"
server_key: "5767271c81dd9715eb33cee2477d8fd6"
server_key_iv: "c071bd62ed91fede2d046c6e86d692f9"
server_key_version: "encryptionkey@0"
I20260808 14:38:46.080494 20578 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 14:38:46.083173 20594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:46.084537 20578 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:38:46.084649 20578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "895cfff4c0bd4e27884296899b584d5d"
format_stamp: "Formatted at 2026-08-08 14:38:46 on dist-test-slave-p3t5"
server_key: "5767271c81dd9715eb33cee2477d8fd6"
server_key_iv: "c071bd62ed91fede2d046c6e86d692f9"
server_key_version: "encryptionkey@0"
I20260808 14:38:46.084748 20578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:46.122844 20578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:46.126399 20578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:46.126581 20578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:46.134618 20578 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:39787
I20260808 14:38:46.134642 20646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:39787 every 8 connection(s)
I20260808 14:38:46.135695 20578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:38:46.138779 20647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:46.141690 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 20578
I20260808 14:38:46.141861 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:38:46.142159 20033 external_mini_cluster.cc:1428] Setting key 7d4d0d36abf7bd3fc119e4c86d57a5fc
I20260808 14:38:46.145387 20647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d: Bootstrap starting.
I20260808 14:38:46.147832 20647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:46.148633 20647 log.cc:824] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d: Log is configured to *not* fsync() on all Append() calls
I20260808 14:38:46.150858 20647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d: No bootstrap required, opened a new log
Aug 08 14:38:46 dist-test-slave-p3t5 krb5kdc[20565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:46.154980 20647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "895cfff4c0bd4e27884296899b584d5d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39787 } }
I20260808 14:38:46.155272 20647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:46.155346 20647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 895cfff4c0bd4e27884296899b584d5d, State: Initialized, Role: FOLLOWER
I20260808 14:38:46.155874 20647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "895cfff4c0bd4e27884296899b584d5d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39787 } }
I20260808 14:38:46.156065 20647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:38:46.156143 20647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:38:46.156301 20647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:46.157567 20647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "895cfff4c0bd4e27884296899b584d5d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39787 } }
I20260808 14:38:46.158001 20647 leader_election.cc:302] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 895cfff4c0bd4e27884296899b584d5d; no voters:
I20260808 14:38:46.158366 20647 leader_election.cc:288] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:38:46.158486 20652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:38:46.158628 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:46.143754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39562 (local address 127.19.144.126:39787)
0808 14:38:46.144283 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:46.144300 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:46.144341 (+ 41us) server_negotiation.cc:407] Connection header received
0808 14:38:46.144990 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:46.145008 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:46.145377 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:46.145818 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:46.146705 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:46.147765 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:46.148719 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:46.149268 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:46.149446 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:46.152442 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:46.152448 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:46.152463 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:46.152495 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:46.155116 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:46.155684 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:46.155692 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:46.155698 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:46.155870 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:46.156193 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:46.156196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:46.156197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:46.156720 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:46.156910 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:46.157179 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 14:38:46.157517 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":159,"threads_started":1}
I20260808 14:38:46.158744 20652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [term 1 LEADER]: Becoming Leader. State: Replica: 895cfff4c0bd4e27884296899b584d5d, State: Running, Role: LEADER
I20260808 14:38:46.159775 20647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:38:46.160218 20652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "895cfff4c0bd4e27884296899b584d5d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39787 } }
I20260808 14:38:46.162400 20653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "895cfff4c0bd4e27884296899b584d5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "895cfff4c0bd4e27884296899b584d5d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39787 } } }
I20260808 14:38:46.162510 20653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [sys.catalog]: This master's current role is: LEADER
I20260808 14:38:46.162102 20654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 895cfff4c0bd4e27884296899b584d5d. Latest consensus state: current_term: 1 leader_uuid: "895cfff4c0bd4e27884296899b584d5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "895cfff4c0bd4e27884296899b584d5d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39787 } } }
I20260808 14:38:46.162878 20654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d [sys.catalog]: This master's current role is: LEADER
W20260808 14:38:46.165051 20666 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:38:46.165154 20666 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:38:46.165230 20667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:38:46.166076 20667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:38:46.172396 20667 catalog_manager.cc:1347] Generated new cluster ID: 5ffe5b52516e47d291a4f48463ee4ee6
I20260808 14:38:46.172494 20667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:38:46.187929 20667 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:38:46.189091 20667 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:38:46.200485 20667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 895cfff4c0bd4e27884296899b584d5d: Generated new TSK 0
I20260808 14:38:46.201452 20667 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:38:46.241454 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:39787
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:46.363302 20673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:46.363595 20673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:46.363641 20673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:46.367856 20673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:46.367926 20673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:46.367988 20673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:38:46.373402 20673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:39787
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:46.374651 20673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:46.375732 20673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:46.386240 20678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:46.386240 20679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:46.386253 20681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:46.386562 20673 server_base.cc:1034] running on GCE node
I20260808 14:38:46.387070 20673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:46.387852 20673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:46.389074 20673 hybrid_clock.cc:648] HybridClock initialized: now 1786199926389050 us; error 55 us; skew 500 ppm
Aug 08 14:38:46 dist-test-slave-p3t5 krb5kdc[20565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199926, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:46.392251 20673 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:38:46.393208 20673 webserver.cc:460] Webserver started at http://127.19.144.65:44937/ using document root <none> and password file <none>
I20260808 14:38:46.393898 20673 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:46.393997 20673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:46.394251 20673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:46.396515 20673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "51dd52193d54401da418ab3308555f8a"
format_stamp: "Formatted at 2026-08-08 14:38:46 on dist-test-slave-p3t5"
server_key: "1ca265c1d81287e4be6da265c12cd3bf"
server_key_iv: "1f48393814337448d00544bbe0b5d7d6"
server_key_version: "encryptionkey@0"
I20260808 14:38:46.397185 20673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "51dd52193d54401da418ab3308555f8a"
format_stamp: "Formatted at 2026-08-08 14:38:46 on dist-test-slave-p3t5"
server_key: "1ca265c1d81287e4be6da265c12cd3bf"
server_key_iv: "1f48393814337448d00544bbe0b5d7d6"
server_key_version: "encryptionkey@0"
I20260808 14:38:46.401489 20673 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260808 14:38:46.404286 20689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:46.405669 20673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:38:46.405865 20673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "51dd52193d54401da418ab3308555f8a"
format_stamp: "Formatted at 2026-08-08 14:38:46 on dist-test-slave-p3t5"
server_key: "1ca265c1d81287e4be6da265c12cd3bf"
server_key_iv: "1f48393814337448d00544bbe0b5d7d6"
server_key_version: "encryptionkey@0"
I20260808 14:38:46.406009 20673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:46.446743 20673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:46.450549 20673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:46.450795 20673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:46.451469 20673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:38:46.452425 20673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:38:46.452507 20673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:46.452577 20673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:38:46.452610 20673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:46.462463 20673 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:45129
I20260808 14:38:46.462535 20802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:45129 every 8 connection(s)
I20260808 14:38:46.463511 20673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:38:46.471378 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 20673
I20260808 14:38:46.471503 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:38:46.471804 20033 external_mini_cluster.cc:1428] Setting key 36884febf238adce9447884feb06f995
Aug 08 14:38:46 dist-test-slave-p3t5 krb5kdc[20565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199926, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:46.479460 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:46.465481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:49469 (local address 127.19.144.126:39787)
0808 14:38:46.465689 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:46.465694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:46.466612 (+ 918us) server_negotiation.cc:407] Connection header received
0808 14:38:46.468010 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:46.468014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:46.468079 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:46.468159 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:46.469440 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:46.470083 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:46.471302 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:46.471685 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:46.471829 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:46.474822 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:46.474829 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:46.474836 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:46.474873 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:46.476500 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:46.477204 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:46.477207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:46.477208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:46.477291 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:46.477745 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:46.477748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:46.477749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:46.477959 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:46.478040 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:46.478888 (+ 848us) server_negotiation.cc:299] Negotiation successful
0808 14:38:46.479152 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 14:38:46.480345 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:46.465969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:39787 (local address 127.19.144.65:49469)
0808 14:38:46.466448 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:46.466495 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:46.467767 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:46.468339 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:46.468346 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:46.468730 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:46.469278 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:46.469294 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:46.470211 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:46.470214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:46.471154 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:46.471164 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:46.471904 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:46.471912 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:46.472103 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:46.472821 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:46.472839 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:46.474621 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:46.476632 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:46.476639 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:46.476663 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:46.477096 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:46.477387 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:46.477393 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:46.477398 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:46.477641 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:46.478071 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:46.478082 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:46.478465 (+ 383us) client_negotiation.cc:769] Sending connection context
0808 14:38:46.478732 (+ 267us) client_negotiation.cc:240] Negotiation successful
0808 14:38:46.479162 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":131,"threads_started":1}
I20260808 14:38:46.481855 20803 heartbeater.cc:347] Connected to a master server at 127.19.144.126:39787
I20260808 14:38:46.482156 20803 heartbeater.cc:464] Registering TS with master...
I20260808 14:38:46.482836 20803 heartbeater.cc:511] Master 127.19.144.126:39787 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:38:46.484730 20611 ts_manager.cc:194] Registered new tserver with Master: 51dd52193d54401da418ab3308555f8a (127.19.144.65:45129)
I20260808 14:38:46.485976 20611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:49469
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:38:46.505146 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:39787
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:46.627882 20808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:46.628185 20808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:46.628230 20808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:46.632382 20808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:46.632452 20808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:46.632514 20808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:38:46.637979 20808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:39787
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:46.639133 20808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:46.640193 20808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:46.650723 20814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:46.650727 20816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:46.650730 20813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:46.651057 20808 server_base.cc:1034] running on GCE node
I20260808 14:38:46.651548 20808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:46.652297 20808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:46.653496 20808 hybrid_clock.cc:648] HybridClock initialized: now 1786199926653482 us; error 28 us; skew 500 ppm
Aug 08 14:38:46 dist-test-slave-p3t5 krb5kdc[20565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199926, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:46.656432 20808 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:38:46.657503 20808 webserver.cc:460] Webserver started at http://127.19.144.66:38413/ using document root <none> and password file <none>
I20260808 14:38:46.658154 20808 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:46.658253 20808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:46.658512 20808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:46.660857 20808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "ad5c6ecb1acc4f53bca5f9db3a2a5b8f"
format_stamp: "Formatted at 2026-08-08 14:38:46 on dist-test-slave-p3t5"
server_key: "6ca48097fb1de222f8f81ee49a193d39"
server_key_iv: "75d232fcbd0b4eb927881bf48a1065a2"
server_key_version: "encryptionkey@0"
I20260808 14:38:46.661705 20808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "ad5c6ecb1acc4f53bca5f9db3a2a5b8f"
format_stamp: "Formatted at 2026-08-08 14:38:46 on dist-test-slave-p3t5"
server_key: "6ca48097fb1de222f8f81ee49a193d39"
server_key_iv: "75d232fcbd0b4eb927881bf48a1065a2"
server_key_version: "encryptionkey@0"
I20260808 14:38:46.665979 20808 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:38:46.668541 20823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:46.669936 20808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:38:46.670047 20808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "ad5c6ecb1acc4f53bca5f9db3a2a5b8f"
format_stamp: "Formatted at 2026-08-08 14:38:46 on dist-test-slave-p3t5"
server_key: "6ca48097fb1de222f8f81ee49a193d39"
server_key_iv: "75d232fcbd0b4eb927881bf48a1065a2"
server_key_version: "encryptionkey@0"
I20260808 14:38:46.670150 20808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:46.681313 20808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:46.684544 20808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:46.684715 20808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:46.685294 20808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:38:46.686297 20808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:38:46.686349 20808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:46.686393 20808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:38:46.686409 20808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:46.695637 20808 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:43287
I20260808 14:38:46.695708 20937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:43287 every 8 connection(s)
I20260808 14:38:46.696681 20808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:38:46.703531 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 20808
I20260808 14:38:46.703651 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:38:46.703923 20033 external_mini_cluster.cc:1428] Setting key 468eaabdd137c808d2d234ceb0331713
Aug 08 14:38:46 dist-test-slave-p3t5 krb5kdc[20565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199926, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:46.713493 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:46.698962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:59401 (local address 127.19.144.126:39787)
0808 14:38:46.699133 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:46.699137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:46.700085 (+ 948us) server_negotiation.cc:407] Connection header received
0808 14:38:46.701516 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:46.701520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:46.701580 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:46.701707 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:46.703083 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:46.703888 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:46.705692 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:46.706036 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:46.706174 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:46.708847 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:46.708853 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:46.708856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:46.708889 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:46.710727 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:46.711521 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:46.711527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:46.711531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:46.711638 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:46.712026 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:46.712030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:46.712032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:46.712349 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:46.712480 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:46.713117 (+ 637us) server_negotiation.cc:299] Negotiation successful
0808 14:38:46.713292 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 14:38:46.714463 20940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:46.699364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:39787 (local address 127.19.144.66:59401)
0808 14:38:46.699893 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:46.699959 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:46.701261 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:46.701894 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:46.701902 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:46.702339 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:46.702921 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:46.702936 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:46.704078 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:46.704081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:46.705522 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:46.705538 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:46.706202 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:46.706205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:46.706376 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:46.707071 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:46.707089 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:46.708638 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:46.710876 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:46.710886 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:46.710926 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:46.711398 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:46.711748 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:46.711750 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:46.711752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:46.711915 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:46.712522 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:46.712529 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:46.712824 (+ 295us) client_negotiation.cc:769] Sending connection context
0808 14:38:46.713021 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 14:38:46.713350 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":137,"threads_started":1}
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
I20260808 14:38:46.716024 20938 heartbeater.cc:347] Connected to a master server at 127.19.144.126:39787
I20260808 14:38:46.716322 20938 heartbeater.cc:464] Registering TS with master...
I20260808 14:38:46.716907 20938 heartbeater.cc:511] Master 127.19.144.126:39787 requested a full tablet report, sending...
I20260808 14:38:46.718335 20611 ts_manager.cc:194] Registered new tserver with Master: ad5c6ecb1acc4f53bca5f9db3a2a5b8f (127.19.144.66:43287)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
I20260808 14:38:46.719123 20611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:59401
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:38:46.737349 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:39787
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:46.859681 20943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:46.860038 20943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:46.860137 20943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:46.864753 20943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:46.864897 20943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:46.865008 20943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:38:46.870667 20943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:39787
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:46.871901 20943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:46.873018 20943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:46.883397 20951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:46.883435 20948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:46.883397 20949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:46.883679 20943 server_base.cc:1034] running on GCE node
I20260808 14:38:46.884217 20943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:46.884936 20943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:46.886132 20943 hybrid_clock.cc:648] HybridClock initialized: now 1786199926886092 us; error 51 us; skew 500 ppm
Aug 08 14:38:46 dist-test-slave-p3t5 krb5kdc[20565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199926, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:46.889178 20943 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:38:46.890268 20943 webserver.cc:460] Webserver started at http://127.19.144.67:40815/ using document root <none> and password file <none>
I20260808 14:38:46.890893 20943 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:46.890991 20943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:46.891232 20943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:46.893455 20943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "fdcc5434bbc845609fdd89b54859a656"
format_stamp: "Formatted at 2026-08-08 14:38:46 on dist-test-slave-p3t5"
server_key: "6d66c58ba9f3a642471f551338f35a04"
server_key_iv: "d79340f2e529c2e86492b269bdeb0ca6"
server_key_version: "encryptionkey@0"
I20260808 14:38:46.894117 20943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "fdcc5434bbc845609fdd89b54859a656"
format_stamp: "Formatted at 2026-08-08 14:38:46 on dist-test-slave-p3t5"
server_key: "6d66c58ba9f3a642471f551338f35a04"
server_key_iv: "d79340f2e529c2e86492b269bdeb0ca6"
server_key_version: "encryptionkey@0"
I20260808 14:38:46.898224 20943 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:38:46.900928 20958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:46.902396 20943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:38:46.902568 20943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "fdcc5434bbc845609fdd89b54859a656"
format_stamp: "Formatted at 2026-08-08 14:38:46 on dist-test-slave-p3t5"
server_key: "6d66c58ba9f3a642471f551338f35a04"
server_key_iv: "d79340f2e529c2e86492b269bdeb0ca6"
server_key_version: "encryptionkey@0"
I20260808 14:38:46.902704 20943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:46.940497 20943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:46.944203 20943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:46.944471 20943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:46.945165 20943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:38:46.946138 20943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:38:46.946213 20943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:46.946300 20943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:38:46.946352 20943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:46.955883 20943 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:41171
I20260808 14:38:46.955904 21072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:41171 every 8 connection(s)
I20260808 14:38:46.956879 20943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:38:46.965903 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 20943
I20260808 14:38:46.966054 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:38:46.966355 20033 external_mini_cluster.cc:1428] Setting key 474cefa183d98c686d357f3912d9702e
Aug 08 14:38:46 dist-test-slave-p3t5 krb5kdc[20565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199926, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:46.972379 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:46.958937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:34439 (local address 127.19.144.126:39787)
0808 14:38:46.959130 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:46.959135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:46.960104 (+ 969us) server_negotiation.cc:407] Connection header received
0808 14:38:46.961225 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:46.961228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:46.961279 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:46.961362 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:46.962852 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:46.963391 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:46.964788 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:46.965048 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:46.965141 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:46.968583 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:46.968586 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:46.968588 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:46.968615 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:46.970247 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:46.970817 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:46.970820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:46.970821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:46.970906 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:46.971288 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:46.971290 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:46.971291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:46.971483 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:46.971564 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:46.972124 (+ 560us) server_negotiation.cc:299] Negotiation successful
0808 14:38:46.972238 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 14:38:46.973301 21075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:46.959368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:39787 (local address 127.19.144.67:34439)
0808 14:38:46.959935 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:46.959985 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:46.961012 (+ 1027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:46.961547 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:46.961554 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:46.962116 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:46.962657 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:46.962672 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:46.963558 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:46.963562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:46.964603 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:46.964614 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:46.965213 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:46.965217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:46.965384 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:46.966179 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:46.966198 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:46.968390 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:46.970391 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:46.970397 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:46.970411 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:46.970700 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:46.971022 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:46.971025 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:46.971026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:46.971183 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:46.971607 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:46.971613 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:46.971882 (+ 269us) client_negotiation.cc:769] Sending connection context
0808 14:38:46.972056 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 14:38:46.972363 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":128,"threads_started":1}
I20260808 14:38:46.974622 21073 heartbeater.cc:347] Connected to a master server at 127.19.144.126:39787
I20260808 14:38:46.974927 21073 heartbeater.cc:464] Registering TS with master...
I20260808 14:38:46.975464 21073 heartbeater.cc:511] Master 127.19.144.126:39787 requested a full tablet report, sending...
I20260808 14:38:46.976583 20611 ts_manager.cc:194] Registered new tserver with Master: fdcc5434bbc845609fdd89b54859a656 (127.19.144.67:41171)
I20260808 14:38:46.977171 20611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:34439
I20260808 14:38:46.981324 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:38:46.992849 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:46.985026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39564 (local address 127.19.144.126:39787)
0808 14:38:46.985201 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:46.985206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:46.985376 (+ 170us) server_negotiation.cc:407] Connection header received
0808 14:38:46.985552 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:46.985555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:46.985608 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:46.985726 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:46.986507 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:46.987059 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:46.987928 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:46.988165 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:46.988285 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:46.989761 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:46.989765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:46.989768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:46.989804 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:46.991039 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:46.991500 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:46.991504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:46.991506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:46.991598 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:46.991895 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:46.991899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:46.991900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:46.992148 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:46.992307 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:46.992553 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 14:38:46.992680 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:38:46.998157 20611 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:38:47.001174 20611 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:38:47.017561 21087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:47.010802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:43287 (local address 127.0.0.1:48368)
0808 14:38:47.011550 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:47.011564 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:47.011657 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:47.012667 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:47.012671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:47.012717 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:47.013030 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:47.013038 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.014366 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.014373 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:47.016005 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:47.016017 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.016890 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.016901 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:47.017011 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:47.017060 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 14:38:47.017206 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 14:38:47.017383 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":194,"spinlock_wait_cycles":9088,"thread_start_us":118,"threads_started":1}
I20260808 14:38:47.017674 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:47.011620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48368 (local address 127.19.144.66:43287)
0808 14:38:47.012183 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:47.012198 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:47.012219 (+ 21us) server_negotiation.cc:407] Connection header received
0808 14:38:47.012297 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:47.012303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:47.012473 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:47.013312 (+ 839us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:47.013450 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.014226 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.016160 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.016751 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.016889 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:47.017061 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:47.017373 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 14:38:47.017494 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":105,"threads_started":1}
I20260808 14:38:47.017705 21088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:47.011568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32802 (local address 127.19.144.65:45129)
0808 14:38:47.012193 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:47.012200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:47.012218 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:38:47.012283 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:47.012287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:47.012480 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:47.012838 (+ 358us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:47.013163 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.014473 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.016147 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.016926 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.017187 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:47.017354 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:47.017419 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 14:38:47.017522 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":75,"threads_started":1}
I20260808 14:38:47.017768 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:47.011640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48744 (local address 127.19.144.67:41171)
0808 14:38:47.012072 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:47.012084 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:47.012110 (+ 26us) server_negotiation.cc:407] Connection header received
0808 14:38:47.012194 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:47.012200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:47.012659 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:47.012874 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:47.013308 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.014474 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.015967 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.016834 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.017346 (+ 512us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:47.017478 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:47.017522 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 14:38:47.017601 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":240,"threads_started":1}
I20260808 14:38:47.018287 21085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:47.010732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:41171 (local address 127.0.0.1:48744)
0808 14:38:47.011122 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:47.011172 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:47.011316 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:47.012803 (+ 1487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:47.012806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:47.012820 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:47.013197 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:47.013202 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.014647 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.014651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:47.015831 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:47.015847 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.016987 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.016993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:47.017114 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:47.017187 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 14:38:47.017240 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 14:38:47.018182 (+ 942us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":1099,"thread_start_us":129,"threads_started":1}
I20260808 14:38:47.018635 21086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:47.011097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:45129 (local address 127.0.0.1:32802)
0808 14:38:47.011837 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:47.011849 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:47.011941 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:47.012665 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:47.012673 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:47.012715 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:47.013030 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:47.013038 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.014612 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.014615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:47.016009 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:47.016017 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.017046 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.017049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:47.017206 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:47.017231 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 14:38:47.017286 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 14:38:47.018488 (+ 1202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":1399,"spinlock_wait_cycles":9088,"thread_start_us":93,"threads_started":1}
I20260808 14:38:47.021411 21007 tablet_service.cc:1467] Processing CreateTablet for tablet 096f31e906e54d60b45087b64e8f3676 (DEFAULT_TABLE table=test-table [id=d29e1454842a46f3990c05e0d5b0d92e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:47.021481 20737 tablet_service.cc:1467] Processing CreateTablet for tablet 096f31e906e54d60b45087b64e8f3676 (DEFAULT_TABLE table=test-table [id=d29e1454842a46f3990c05e0d5b0d92e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:47.021485 20872 tablet_service.cc:1467] Processing CreateTablet for tablet 096f31e906e54d60b45087b64e8f3676 (DEFAULT_TABLE table=test-table [id=d29e1454842a46f3990c05e0d5b0d92e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:47.022851 20737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 096f31e906e54d60b45087b64e8f3676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:47.022874 20872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 096f31e906e54d60b45087b64e8f3676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:47.023263 21007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 096f31e906e54d60b45087b64e8f3676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:47.030577 21091 tablet_bootstrap.cc:492] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656: Bootstrap starting.
I20260808 14:38:47.031961 21092 tablet_bootstrap.cc:492] T 096f31e906e54d60b45087b64e8f3676 P 51dd52193d54401da418ab3308555f8a: Bootstrap starting.
I20260808 14:38:47.032968 21091 tablet_bootstrap.cc:654] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:47.034039 21092 tablet_bootstrap.cc:654] T 096f31e906e54d60b45087b64e8f3676 P 51dd52193d54401da418ab3308555f8a: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:47.034349 21091 log.cc:824] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656: Log is configured to *not* fsync() on all Append() calls
I20260808 14:38:47.034906 21092 log.cc:824] T 096f31e906e54d60b45087b64e8f3676 P 51dd52193d54401da418ab3308555f8a: Log is configured to *not* fsync() on all Append() calls
I20260808 14:38:47.035718 21093 tablet_bootstrap.cc:492] T 096f31e906e54d60b45087b64e8f3676 P ad5c6ecb1acc4f53bca5f9db3a2a5b8f: Bootstrap starting.
I20260808 14:38:47.036497 21091 tablet_bootstrap.cc:492] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656: No bootstrap required, opened a new log
I20260808 14:38:47.036722 21091 ts_tablet_manager.cc:1397] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 14:38:47.037061 21092 tablet_bootstrap.cc:492] T 096f31e906e54d60b45087b64e8f3676 P 51dd52193d54401da418ab3308555f8a: No bootstrap required, opened a new log
I20260808 14:38:47.037259 21092 ts_tablet_manager.cc:1397] T 096f31e906e54d60b45087b64e8f3676 P 51dd52193d54401da418ab3308555f8a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 14:38:47.038571 21093 tablet_bootstrap.cc:654] T 096f31e906e54d60b45087b64e8f3676 P ad5c6ecb1acc4f53bca5f9db3a2a5b8f: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:47.039603 21093 log.cc:824] T 096f31e906e54d60b45087b64e8f3676 P ad5c6ecb1acc4f53bca5f9db3a2a5b8f: Log is configured to *not* fsync() on all Append() calls
I20260808 14:38:47.040031 21091 raft_consensus.cc:348] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdcc5434bbc845609fdd89b54859a656" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41171 } } peers { permanent_uuid: "ad5c6ecb1acc4f53bca5f9db3a2a5b8f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43287 } } peers { permanent_uuid: "51dd52193d54401da418ab3308555f8a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 45129 } }
I20260808 14:38:47.040370 21091 raft_consensus.cc:374] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:47.040486 21091 raft_consensus.cc:729] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdcc5434bbc845609fdd89b54859a656, State: Initialized, Role: FOLLOWER
I20260808 14:38:47.041028 21091 consensus_queue.cc:260] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdcc5434bbc845609fdd89b54859a656" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41171 } } peers { permanent_uuid: "ad5c6ecb1acc4f53bca5f9db3a2a5b8f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43287 } } peers { permanent_uuid: "51dd52193d54401da418ab3308555f8a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 45129 } }
I20260808 14:38:47.041941 21093 tablet_bootstrap.cc:492] T 096f31e906e54d60b45087b64e8f3676 P ad5c6ecb1acc4f53bca5f9db3a2a5b8f: No bootstrap required, opened a new log
I20260808 14:38:47.042214 21091 ts_tablet_manager.cc:1428] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260808 14:38:47.042203 21093 ts_tablet_manager.cc:1397] T 096f31e906e54d60b45087b64e8f3676 P ad5c6ecb1acc4f53bca5f9db3a2a5b8f: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 14:38:47.042114 21092 raft_consensus.cc:348] T 096f31e906e54d60b45087b64e8f3676 P 51dd52193d54401da418ab3308555f8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdcc5434bbc845609fdd89b54859a656" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41171 } } peers { permanent_uuid: "ad5c6ecb1acc4f53bca5f9db3a2a5b8f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43287 } } peers { permanent_uuid: "51dd52193d54401da418ab3308555f8a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 45129 } }
I20260808 14:38:47.042393 21092 raft_consensus.cc:374] T 096f31e906e54d60b45087b64e8f3676 P 51dd52193d54401da418ab3308555f8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:47.042466 21092 raft_consensus.cc:729] T 096f31e906e54d60b45087b64e8f3676 P 51dd52193d54401da418ab3308555f8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51dd52193d54401da418ab3308555f8a, State: Initialized, Role: FOLLOWER
I20260808 14:38:47.042658 21073 heartbeater.cc:503] Master 127.19.144.126:39787 was elected leader, sending a full tablet report...
I20260808 14:38:47.042935 21092 consensus_queue.cc:260] T 096f31e906e54d60b45087b64e8f3676 P 51dd52193d54401da418ab3308555f8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdcc5434bbc845609fdd89b54859a656" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41171 } } peers { permanent_uuid: "ad5c6ecb1acc4f53bca5f9db3a2a5b8f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43287 } } peers { permanent_uuid: "51dd52193d54401da418ab3308555f8a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 45129 } }
I20260808 14:38:47.043646 20803 heartbeater.cc:503] Master 127.19.144.126:39787 was elected leader, sending a full tablet report...
I20260808 14:38:47.043952 21092 ts_tablet_manager.cc:1428] T 096f31e906e54d60b45087b64e8f3676 P 51dd52193d54401da418ab3308555f8a: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260808 14:38:47.047575 21093 raft_consensus.cc:348] T 096f31e906e54d60b45087b64e8f3676 P ad5c6ecb1acc4f53bca5f9db3a2a5b8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdcc5434bbc845609fdd89b54859a656" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41171 } } peers { permanent_uuid: "ad5c6ecb1acc4f53bca5f9db3a2a5b8f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43287 } } peers { permanent_uuid: "51dd52193d54401da418ab3308555f8a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 45129 } }
I20260808 14:38:47.047851 21093 raft_consensus.cc:374] T 096f31e906e54d60b45087b64e8f3676 P ad5c6ecb1acc4f53bca5f9db3a2a5b8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:47.047935 21093 raft_consensus.cc:729] T 096f31e906e54d60b45087b64e8f3676 P ad5c6ecb1acc4f53bca5f9db3a2a5b8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5c6ecb1acc4f53bca5f9db3a2a5b8f, State: Initialized, Role: FOLLOWER
I20260808 14:38:47.048396 21093 consensus_queue.cc:260] T 096f31e906e54d60b45087b64e8f3676 P ad5c6ecb1acc4f53bca5f9db3a2a5b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdcc5434bbc845609fdd89b54859a656" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41171 } } peers { permanent_uuid: "ad5c6ecb1acc4f53bca5f9db3a2a5b8f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43287 } } peers { permanent_uuid: "51dd52193d54401da418ab3308555f8a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 45129 } }
I20260808 14:38:47.049118 20938 heartbeater.cc:503] Master 127.19.144.126:39787 was elected leader, sending a full tablet report...
I20260808 14:38:47.049320 21093 ts_tablet_manager.cc:1428] T 096f31e906e54d60b45087b64e8f3676 P ad5c6ecb1acc4f53bca5f9db3a2a5b8f: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 14:38:47.062346 21097 raft_consensus.cc:482] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:38:47.062600 21097 raft_consensus.cc:504] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdcc5434bbc845609fdd89b54859a656" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41171 } } peers { permanent_uuid: "ad5c6ecb1acc4f53bca5f9db3a2a5b8f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43287 } } peers { permanent_uuid: "51dd52193d54401da418ab3308555f8a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 45129 } }
I20260808 14:38:47.063850 21097 leader_election.cc:288] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad5c6ecb1acc4f53bca5f9db3a2a5b8f (127.19.144.66:43287), 51dd52193d54401da418ab3308555f8a (127.19.144.65:45129)
I20260808 14:38:47.067710 21075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:47.064061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:43287 (local address 127.19.144.67:37753)
0808 14:38:47.064441 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:47.064459 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:47.064570 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:47.064833 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:47.064836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:47.064853 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:47.064986 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:47.064991 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.065774 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.065778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:47.066576 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:47.066585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.067244 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.067248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:47.067362 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:47.067385 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 14:38:47.067486 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 14:38:47.067574 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303}
I20260808 14:38:47.067710 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:47.064190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:37753 (local address 127.19.144.66:43287)
0808 14:38:47.064369 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:47.064376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:47.064534 (+ 158us) server_negotiation.cc:407] Connection header received
0808 14:38:47.064680 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:47.064685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:47.064732 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:47.064820 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:47.065097 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.065660 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.066705 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.067120 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.067252 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:47.067318 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:47.067503 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 14:38:47.067574 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 14:38:47.067956 21088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:47.064288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:44019 (local address 127.19.144.65:45129)
0808 14:38:47.064432 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:47.064437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:47.064692 (+ 255us) server_negotiation.cc:407] Connection header received
0808 14:38:47.064854 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:47.064859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:47.064907 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:47.064988 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:47.065278 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.065879 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.066938 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.067366 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.067503 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:47.067556 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:47.067745 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 14:38:47.067830 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:38:47.068212 21100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:47.064150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:45129 (local address 127.19.144.67:44019)
0808 14:38:47.064580 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:47.064592 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:47.064727 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:47.065012 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:47.065015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:47.065032 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:47.065159 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:47.065164 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.065996 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.066001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:47.066809 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:47.066818 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.067491 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.067495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:47.067609 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:47.067638 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 14:38:47.067727 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 14:38:47.067798 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":1,"thread_start_us":160,"threads_started":1}
I20260808 14:38:47.068648 20892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "096f31e906e54d60b45087b64e8f3676" candidate_uuid: "fdcc5434bbc845609fdd89b54859a656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5c6ecb1acc4f53bca5f9db3a2a5b8f" is_pre_election: true
I20260808 14:38:47.068914 20757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "096f31e906e54d60b45087b64e8f3676" candidate_uuid: "fdcc5434bbc845609fdd89b54859a656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51dd52193d54401da418ab3308555f8a" is_pre_election: true
I20260808 14:38:47.069005 20892 raft_consensus.cc:2454] T 096f31e906e54d60b45087b64e8f3676 P ad5c6ecb1acc4f53bca5f9db3a2a5b8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdcc5434bbc845609fdd89b54859a656 in term 0.
I20260808 14:38:47.069226 20757 raft_consensus.cc:2454] T 096f31e906e54d60b45087b64e8f3676 P 51dd52193d54401da418ab3308555f8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdcc5434bbc845609fdd89b54859a656 in term 0.
I20260808 14:38:47.069587 20962 leader_election.cc:302] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad5c6ecb1acc4f53bca5f9db3a2a5b8f, fdcc5434bbc845609fdd89b54859a656; no voters:
I20260808 14:38:47.069957 21097 raft_consensus.cc:2781] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:38:47.070040 21097 raft_consensus.cc:482] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:38:47.070071 21097 raft_consensus.cc:3037] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:47.071182 21097 raft_consensus.cc:504] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdcc5434bbc845609fdd89b54859a656" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41171 } } peers { permanent_uuid: "ad5c6ecb1acc4f53bca5f9db3a2a5b8f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43287 } } peers { permanent_uuid: "51dd52193d54401da418ab3308555f8a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 45129 } }
I20260808 14:38:47.071566 21097 leader_election.cc:288] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [CANDIDATE]: Term 1 election: Requested vote from peers ad5c6ecb1acc4f53bca5f9db3a2a5b8f (127.19.144.66:43287), 51dd52193d54401da418ab3308555f8a (127.19.144.65:45129)
I20260808 14:38:47.072036 20892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "096f31e906e54d60b45087b64e8f3676" candidate_uuid: "fdcc5434bbc845609fdd89b54859a656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5c6ecb1acc4f53bca5f9db3a2a5b8f"
I20260808 14:38:47.072101 20757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "096f31e906e54d60b45087b64e8f3676" candidate_uuid: "fdcc5434bbc845609fdd89b54859a656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51dd52193d54401da418ab3308555f8a"
I20260808 14:38:47.072188 20892 raft_consensus.cc:3037] T 096f31e906e54d60b45087b64e8f3676 P ad5c6ecb1acc4f53bca5f9db3a2a5b8f [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:47.072275 20757 raft_consensus.cc:3037] T 096f31e906e54d60b45087b64e8f3676 P 51dd52193d54401da418ab3308555f8a [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:47.073673 20892 raft_consensus.cc:2454] T 096f31e906e54d60b45087b64e8f3676 P ad5c6ecb1acc4f53bca5f9db3a2a5b8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdcc5434bbc845609fdd89b54859a656 in term 1.
I20260808 14:38:47.073691 20757 raft_consensus.cc:2454] T 096f31e906e54d60b45087b64e8f3676 P 51dd52193d54401da418ab3308555f8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdcc5434bbc845609fdd89b54859a656 in term 1.
I20260808 14:38:47.074000 20962 leader_election.cc:302] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad5c6ecb1acc4f53bca5f9db3a2a5b8f, fdcc5434bbc845609fdd89b54859a656; no voters:
I20260808 14:38:47.074239 21097 raft_consensus.cc:2781] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:38:47.074463 21097 raft_consensus.cc:686] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [term 1 LEADER]: Becoming Leader. State: Replica: fdcc5434bbc845609fdd89b54859a656, State: Running, Role: LEADER
I20260808 14:38:47.074816 21097 consensus_queue.cc:237] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdcc5434bbc845609fdd89b54859a656" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41171 } } peers { permanent_uuid: "ad5c6ecb1acc4f53bca5f9db3a2a5b8f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43287 } } peers { permanent_uuid: "51dd52193d54401da418ab3308555f8a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 45129 } }
I20260808 14:38:47.078380 20611 catalog_manager.cc:5495] T 096f31e906e54d60b45087b64e8f3676 P fdcc5434bbc845609fdd89b54859a656 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdcc5434bbc845609fdd89b54859a656 (127.19.144.67). New cstate: current_term: 1 leader_uuid: "fdcc5434bbc845609fdd89b54859a656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdcc5434bbc845609fdd89b54859a656" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5c6ecb1acc4f53bca5f9db3a2a5b8f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51dd52193d54401da418ab3308555f8a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 45129 } health_report { overall_health: UNKNOWN } } }
Aug 08 14:38:47 dist-test-slave-p3t5 krb5kdc[20565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.65@KRBTEST.COM
I20260808 14:38:47.101933 21088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:47.091788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32808 (local address 127.19.144.65:45129)
0808 14:38:47.091985 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:47.091991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:47.092180 (+ 189us) server_negotiation.cc:407] Connection header received
0808 14:38:47.092383 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:47.092386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:47.092431 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:47.092526 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:47.093552 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.094260 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.095153 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.095409 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.095511 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:47.098294 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:47.098310 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:47.098328 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:47.098362 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:47.100126 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:47.100568 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:47.100576 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:47.100577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:47.100673 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:47.100912 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:47.100915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:47.100916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:47.101187 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:47.101373 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:47.101530 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 14:38:47.101724 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Aug 08 14:38:47 dist-test-slave-p3t5 krb5kdc[20565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199927, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 14:38:47 dist-test-slave-p3t5 krb5kdc[20565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199927, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.144.65@KRBTEST.COM
I20260808 14:38:47.133148 21088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:47.123820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32822 (local address 127.19.144.65:45129)
0808 14:38:47.123991 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:47.123997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:47.124180 (+ 183us) server_negotiation.cc:407] Connection header received
0808 14:38:47.124382 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:47.124385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:47.124434 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:47.124528 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:47.125254 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.125855 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.126713 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.126961 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.127076 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:47.129682 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:47.129703 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:47.129709 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:47.129746 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:47.131306 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:47.131758 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:47.131763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:47.131765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:47.131856 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:47.132165 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:47.132168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:47.132170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:47.132361 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:47.132506 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:47.132827 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 14:38:47.132974 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 14:38:47 dist-test-slave-p3t5 krb5kdc[20565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 14:38:47 dist-test-slave-p3t5 krb5kdc[20565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.65@KRBTEST.COM
I20260808 14:38:47.164458 21088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:47.154572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32834 (local address 127.19.144.65:45129)
0808 14:38:47.154753 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:47.154758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:47.154933 (+ 175us) server_negotiation.cc:407] Connection header received
0808 14:38:47.155124 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:47.155127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:47.155181 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:47.155281 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:47.156016 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.156540 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.157407 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:47.157722 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:47.157838 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:47.160678 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:47.160682 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:47.160684 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:47.160714 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:47.162401 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:47.162962 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:47.162965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:47.162967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:47.163060 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:47.163459 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:47.163462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:47.163463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:47.163680 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:47.163794 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:47.164107 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 14:38:47.164229 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 14:38:47.166244 20717 tablet_service.cc:2908] Rejecting scan request for tablet 096f31e906e54d60b45087b64e8f3676: Uninitialized: clean time has not yet been initialized
I20260808 14:38:47.168649 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 20673
I20260808 14:38:47.176837 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 20808
I20260808 14:38:47.184590 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 20943
I20260808 14:38:47.191996 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 20578
2026-08-08T14:38:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2489 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:38:47 dist-test-slave-p3t5 krb5kdc[21115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:38:47 dist-test-slave-p3t5 krb5kdc[21115](info): set up 2 sockets
Aug 08 14:38:47 dist-test-slave-p3t5 krb5kdc[21115](info): commencing operation
krb5kdc: starting...
W20260808 14:38:48.290717 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:38:48 dist-test-slave-p3t5 krb5kdc[21115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:38:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:38:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:38:48.433987 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:45381
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:44785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:45381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:48.553684 21128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:48.554006 21128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:48.554049 21128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:48.559149 21128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:38:48.559253 21128 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 14:38:48.559285 21128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:48.559319 21128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:48.559361 21128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:38:48.559396 21128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:38:48.567946 21128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:44785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:45381
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:45381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:48.569191 21128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:48.570338 21128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:48.579620 21136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:48.579640 21134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:48.579685 21128 server_base.cc:1034] running on GCE node
W20260808 14:38:48.579620 21133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:48.580583 21128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:48.581835 21128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:48.583039 21128 hybrid_clock.cc:648] HybridClock initialized: now 1786199928583026 us; error 30 us; skew 500 ppm
Aug 08 14:38:48 dist-test-slave-p3t5 krb5kdc[21115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199928, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:48.586169 21128 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:38:48.587208 21128 webserver.cc:460] Webserver started at http://127.19.144.126:34501/ using document root <none> and password file <none>
I20260808 14:38:48.587766 21128 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:48.587839 21128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:48.588018 21128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:48.590301 21128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "36441689f9b9466ca80f6d8c41576e0f"
format_stamp: "Formatted at 2026-08-08 14:38:48 on dist-test-slave-p3t5"
server_key: "6ba19fe4bd63548e2347d7b3098fb5da"
server_key_iv: "4dfcee8e7bf9dd965562df15ba6d6027"
server_key_version: "encryptionkey@0"
I20260808 14:38:48.590902 21128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "36441689f9b9466ca80f6d8c41576e0f"
format_stamp: "Formatted at 2026-08-08 14:38:48 on dist-test-slave-p3t5"
server_key: "6ba19fe4bd63548e2347d7b3098fb5da"
server_key_iv: "4dfcee8e7bf9dd965562df15ba6d6027"
server_key_version: "encryptionkey@0"
I20260808 14:38:48.594893 21128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 14:38:48.597482 21143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:48.598850 21128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:38:48.599004 21128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "36441689f9b9466ca80f6d8c41576e0f"
format_stamp: "Formatted at 2026-08-08 14:38:48 on dist-test-slave-p3t5"
server_key: "6ba19fe4bd63548e2347d7b3098fb5da"
server_key_iv: "4dfcee8e7bf9dd965562df15ba6d6027"
server_key_version: "encryptionkey@0"
I20260808 14:38:48.599129 21128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:48.615803 21128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:48.619294 21128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:48.619515 21128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:48.628722 21128 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:45381
I20260808 14:38:48.628728 21207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:45381 every 8 connection(s)
I20260808 14:38:48.629902 21128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:38:48.632094 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 21128
I20260808 14:38:48.632215 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:38:48.632499 20033 external_mini_cluster.cc:1428] Setting key 418bb5ce97497ea4096dfd9923a59ff0
I20260808 14:38:48.633342 21208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:48.640429 21208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f: Bootstrap starting.
Aug 08 14:38:48 dist-test-slave-p3t5 krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:48.643420 21208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:48.644739 21208 log.cc:824] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f: Log is configured to *not* fsync() on all Append() calls
I20260808 14:38:48.647153 21208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f: No bootstrap required, opened a new log
I20260808 14:38:48.647969 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:48.634019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44498 (local address 127.19.144.126:45381)
0808 14:38:48.634449 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:48.634466 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:48.634502 (+ 36us) server_negotiation.cc:407] Connection header received
0808 14:38:48.635162 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:48.635180 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:48.635522 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:48.635851 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:48.636739 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:48.637877 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:48.639100 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:48.639579 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:48.639694 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:48.642719 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:48.642725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:48.642740 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:48.642771 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:48.644753 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:48.645298 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:48.645304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:48.645309 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:48.645434 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:48.645838 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:48.645842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:48.645843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:48.646279 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:48.646519 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:48.646817 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 14:38:48.647127 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":143,"threads_started":1}
I20260808 14:38:48.650547 21208 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36441689f9b9466ca80f6d8c41576e0f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45381 } }
I20260808 14:38:48.650827 21208 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:48.650918 21208 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36441689f9b9466ca80f6d8c41576e0f, State: Initialized, Role: FOLLOWER
I20260808 14:38:48.651374 21208 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36441689f9b9466ca80f6d8c41576e0f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45381 } }
I20260808 14:38:48.651530 21208 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:38:48.651593 21208 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:38:48.651684 21208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:48.652779 21208 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36441689f9b9466ca80f6d8c41576e0f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45381 } }
I20260808 14:38:48.653141 21208 leader_election.cc:302] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36441689f9b9466ca80f6d8c41576e0f; no voters:
I20260808 14:38:48.653506 21208 leader_election.cc:288] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:38:48.653618 21213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:38:48.653908 21213 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [term 1 LEADER]: Becoming Leader. State: Replica: 36441689f9b9466ca80f6d8c41576e0f, State: Running, Role: LEADER
I20260808 14:38:48.654278 21213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36441689f9b9466ca80f6d8c41576e0f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45381 } }
I20260808 14:38:48.654851 21208 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:38:48.656093 21215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36441689f9b9466ca80f6d8c41576e0f. Latest consensus state: current_term: 1 leader_uuid: "36441689f9b9466ca80f6d8c41576e0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36441689f9b9466ca80f6d8c41576e0f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45381 } } }
I20260808 14:38:48.656230 21215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [sys.catalog]: This master's current role is: LEADER
I20260808 14:38:48.656530 21214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36441689f9b9466ca80f6d8c41576e0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36441689f9b9466ca80f6d8c41576e0f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45381 } } }
I20260808 14:38:48.656643 21222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:38:48.656713 21214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f [sys.catalog]: This master's current role is: LEADER
I20260808 14:38:48.660132 21222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:38:48.666404 21222 catalog_manager.cc:1347] Generated new cluster ID: 546435eb55bf4e25b7bfd28671b5d28f
I20260808 14:38:48.666569 21222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:38:48.685680 21222 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:38:48.686931 21222 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:38:48.693481 21222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f: Generated new TSK 0
I20260808 14:38:48.694279 21222 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:38:48.728770 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:45381
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:44785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:48.849200 21234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:48.849506 21234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:48.849552 21234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:48.853650 21234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:48.853720 21234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:48.853760 21234 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:38:48.853832 21234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:38:48.859766 21234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:44785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.144.126:45381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:48.860973 21234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:48.862107 21234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:48.872684 21240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:48.872922 21234 server_base.cc:1034] running on GCE node
W20260808 14:38:48.872687 21239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:48.872701 21243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:48.873481 21234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:48.874205 21234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:48.875387 21234 hybrid_clock.cc:648] HybridClock initialized: now 1786199928875365 us; error 35 us; skew 500 ppm
Aug 08 14:38:48 dist-test-slave-p3t5 krb5kdc[21115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199928, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:48.878458 21234 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:38:48.879492 21234 webserver.cc:460] Webserver started at http://127.19.144.65:46491/ using document root <none> and password file <none>
I20260808 14:38:48.880143 21234 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:48.880234 21234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:48.880537 21234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:48.884104 21234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "7d1e42ebd8534d948044e34753d7553b"
format_stamp: "Formatted at 2026-08-08 14:38:48 on dist-test-slave-p3t5"
server_key: "87b6a9ea7f1a898c6fa9108e499141e2"
server_key_iv: "62cc794d6883dc79d2319338d6681cc1"
server_key_version: "encryptionkey@0"
I20260808 14:38:48.885002 21234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "7d1e42ebd8534d948044e34753d7553b"
format_stamp: "Formatted at 2026-08-08 14:38:48 on dist-test-slave-p3t5"
server_key: "87b6a9ea7f1a898c6fa9108e499141e2"
server_key_iv: "62cc794d6883dc79d2319338d6681cc1"
server_key_version: "encryptionkey@0"
I20260808 14:38:48.889501 21234 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 14:38:48.892406 21250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:48.894007 21234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:38:48.894209 21234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "7d1e42ebd8534d948044e34753d7553b"
format_stamp: "Formatted at 2026-08-08 14:38:48 on dist-test-slave-p3t5"
server_key: "87b6a9ea7f1a898c6fa9108e499141e2"
server_key_iv: "62cc794d6883dc79d2319338d6681cc1"
server_key_version: "encryptionkey@0"
I20260808 14:38:48.894361 21234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:48.915144 21234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:48.918850 21234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:48.919098 21234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:48.920696 21234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:38:48.920778 21234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:48.920845 21234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:38:48.920913 21234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 14:38:48 dist-test-slave-p3t5 krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199928, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:48.932529 21234 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:37901
I20260808 14:38:48.933475 21234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:38:48.933568 21367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:37901 every 8 connection(s)
I20260808 14:38:48.937091 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:48.921922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:46101 (local address 127.19.144.126:45381)
0808 14:38:48.922085 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:48.922089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:48.923227 (+ 1138us) server_negotiation.cc:407] Connection header received
0808 14:38:48.924570 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:48.924574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:48.924644 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:48.924764 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:48.926311 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:48.926939 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:48.929054 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:48.929435 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:48.929540 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:48.933107 (+ 3567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:48.933112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:48.933115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:48.933155 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:48.935081 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:48.935653 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:48.935656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:48.935657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:48.935743 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:48.936084 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:48.936086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:48.936088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:48.936277 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:48.936350 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:48.936841 (+ 491us) server_negotiation.cc:299] Negotiation successful
0808 14:38:48.936964 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 14:38:48.937494 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 21234
I20260808 14:38:48.937646 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:38:48.938017 20033 external_mini_cluster.cc:1428] Setting key ad9c83c05530a3a645833aa463bb6bc8
I20260808 14:38:48.938196 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:48.922442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45381 (local address 127.19.144.65:46101)
0808 14:38:48.923042 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:48.923093 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:48.924296 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:48.924980 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:48.924993 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:48.925461 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:48.926113 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:48.926130 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:48.927151 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:48.927155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:48.928860 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:48.928875 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:48.929647 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:48.929652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:48.929896 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:48.930740 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:48.930769 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:48.932899 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:48.935213 (+ 2314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:48.935219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:48.935231 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:48.935540 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:48.935829 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:48.935831 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:48.935832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:48.935991 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:48.936364 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:48.936369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:48.936620 (+ 251us) client_negotiation.cc:769] Sending connection context
0808 14:38:48.936780 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 14:38:48.937055 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":152,"threads_started":1}
I20260808 14:38:48.939858 21368 heartbeater.cc:347] Connected to a master server at 127.19.144.126:45381
I20260808 14:38:48.940034 21368 heartbeater.cc:464] Registering TS with master...
I20260808 14:38:48.940629 21368 heartbeater.cc:511] Master 127.19.144.126:45381 requested a full tablet report, sending...
I20260808 14:38:48.942528 21162 ts_manager.cc:194] Registered new tserver with Master: 7d1e42ebd8534d948044e34753d7553b (127.19.144.65:37901)
I20260808 14:38:48.943986 21162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:46101
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:38:48.951284 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:48.942741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:50927 (local address 127.19.144.126:45381)
0808 14:38:48.942888 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:48.942893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:48.942908 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:38:48.942947 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:48.942950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:48.943000 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:48.943084 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:48.943657 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:48.944378 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:48.945935 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:48.946264 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:48.946376 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:48.947904 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:48.947907 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:48.947909 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:48.947932 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:48.949430 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:48.949948 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:48.949951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:48.949952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:48.950061 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:48.950391 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:48.950395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:48.950396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:48.950578 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:48.950685 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:48.950970 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 14:38:48.951124 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:38:48.951292 21375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:48.942347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45381 (local address 127.19.144.65:50927)
0808 14:38:48.942716 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:48.942732 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:48.942844 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:48.943129 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:48.943133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:48.943404 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:48.943536 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:48.943543 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:48.944530 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:48.944537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:48.945783 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:48.945799 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:48.946409 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:48.946416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:48.946554 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:48.947163 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:48.947183 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:48.947745 (+ 562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:48.949550 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:48.949554 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:48.949556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:48.949849 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:48.950174 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:48.950178 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:48.950180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:48.950281 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:48.950689 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:48.950693 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:48.950793 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 14:38:48.950912 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 14:38:48.951074 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":65,"thread_start_us":138,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:38:48.970480 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:45381
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:44785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:49.089216 21377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:49.089512 21377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:49.089560 21377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:49.093660 21377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:49.093734 21377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:49.093773 21377 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:38:49.093847 21377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:38:49.099363 21377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:44785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.144.126:45381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:49.100538 21377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:49.101652 21377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:49.112289 21383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:49.112289 21382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:49.112282 21385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:49.112690 21377 server_base.cc:1034] running on GCE node
I20260808 14:38:49.113123 21377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:49.113883 21377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:49.115060 21377 hybrid_clock.cc:648] HybridClock initialized: now 1786199929115040 us; error 29 us; skew 500 ppm
Aug 08 14:38:49 dist-test-slave-p3t5 krb5kdc[21115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199929, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:49.118183 21377 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:38:49.119141 21377 webserver.cc:460] Webserver started at http://127.19.144.66:36831/ using document root <none> and password file <none>
I20260808 14:38:49.119727 21377 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:49.119791 21377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:49.119977 21377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:49.122169 21377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "4d870ceb7bea47058f8be6b9ba840efe"
format_stamp: "Formatted at 2026-08-08 14:38:49 on dist-test-slave-p3t5"
server_key: "4f63f8aa7e0206ef53a8902360db84a0"
server_key_iv: "6fdc2ad59657f7aabc5ac8e53c37a7bb"
server_key_version: "encryptionkey@0"
I20260808 14:38:49.122715 21377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "4d870ceb7bea47058f8be6b9ba840efe"
format_stamp: "Formatted at 2026-08-08 14:38:49 on dist-test-slave-p3t5"
server_key: "4f63f8aa7e0206ef53a8902360db84a0"
server_key_iv: "6fdc2ad59657f7aabc5ac8e53c37a7bb"
server_key_version: "encryptionkey@0"
I20260808 14:38:49.126597 21377 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:38:49.129117 21393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:49.130438 21377 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:38:49.130541 21377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "4d870ceb7bea47058f8be6b9ba840efe"
format_stamp: "Formatted at 2026-08-08 14:38:49 on dist-test-slave-p3t5"
server_key: "4f63f8aa7e0206ef53a8902360db84a0"
server_key_iv: "6fdc2ad59657f7aabc5ac8e53c37a7bb"
server_key_version: "encryptionkey@0"
I20260808 14:38:49.130631 21377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:49.145068 21377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:49.148607 21377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:49.148835 21377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:49.150549 21377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:38:49.150641 21377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:49.150720 21377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:38:49.150763 21377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 14:38:49 dist-test-slave-p3t5 krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199929, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:49.162263 21377 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:35847
I20260808 14:38:49.162359 21510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:35847 every 8 connection(s)
I20260808 14:38:49.163465 21377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:38:49.166985 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.151454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:43691 (local address 127.19.144.126:45381)
0808 14:38:49.151596 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.151601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.152778 (+ 1177us) server_negotiation.cc:407] Connection header received
0808 14:38:49.153995 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.154001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.154069 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.154138 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:49.156251 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.157129 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.159068 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.159419 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.159589 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.162949 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:49.162954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:49.162957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:49.162991 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:49.164945 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:49.165454 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:49.165462 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:49.165463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:49.165553 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:49.165935 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:49.165937 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:49.165938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:49.166148 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:49.166230 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.166737 (+ 507us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.166855 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 14:38:49.167907 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.152072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45381 (local address 127.19.144.66:43691)
0808 14:38:49.152611 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:49.152658 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:49.153700 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:49.154647 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:49.154655 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:49.155268 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:49.156026 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.156049 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.157300 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.157306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.158899 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.158912 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.159619 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.159622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.159776 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.160587 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:49.160614 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:49.162727 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:49.165063 (+ 2336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:49.165067 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:49.165080 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:49.165343 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:49.165685 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:49.165688 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:49.165689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:49.165840 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:49.166243 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:49.166249 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:49.166500 (+ 251us) client_negotiation.cc:769] Sending connection context
0808 14:38:49.166657 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 14:38:49.166962 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":156,"threads_started":1}
I20260808 14:38:49.168570 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 21377
I20260808 14:38:49.168705 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:38:49.169025 20033 external_mini_cluster.cc:1428] Setting key 6549d28054282cc57982ba094af1ae8a
I20260808 14:38:49.169493 21511 heartbeater.cc:347] Connected to a master server at 127.19.144.126:45381
I20260808 14:38:49.169706 21511 heartbeater.cc:464] Registering TS with master...
I20260808 14:38:49.170300 21511 heartbeater.cc:511] Master 127.19.144.126:45381 requested a full tablet report, sending...
I20260808 14:38:49.171428 21160 ts_manager.cc:194] Registered new tserver with Master: 4d870ceb7bea47058f8be6b9ba840efe (127.19.144.66:35847)
I20260808 14:38:49.172191 21160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:43691
I20260808 14:38:49.180516 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.171806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45381 (local address 127.19.144.66:48063)
0808 14:38:49.172204 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:49.172221 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:49.172334 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:49.172594 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:49.172600 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:49.172857 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:49.173020 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.173030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.174090 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.174093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.174937 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.174948 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.175584 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.175587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.175657 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.176311 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:49.176326 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:49.176775 (+ 449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:49.178917 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:49.178920 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:49.178934 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:49.179129 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:49.179456 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:49.179458 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:49.179460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:49.179536 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:49.180090 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:49.180093 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:49.180173 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 14:38:49.180252 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 14:38:49.180369 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":134,"threads_started":1}
I20260808 14:38:49.180676 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.171875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:48063 (local address 127.19.144.126:45381)
0808 14:38:49.171985 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.171989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.172350 (+ 361us) server_negotiation.cc:407] Connection header received
0808 14:38:49.172404 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.172408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.172466 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.172547 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:49.173157 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.173928 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.175069 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.175464 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.175590 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.176933 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:49.176937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:49.176940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:49.176981 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:49.178798 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:49.179239 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:49.179245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:49.179248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:49.179356 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:49.179630 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:49.179636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:49.179639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:49.179985 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:49.180105 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.180352 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.180485 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:38:49.200938 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:45381
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:44785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:49.323086 21521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:49.323433 21521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:49.323536 21521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:49.327878 21521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:49.327975 21521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:49.328054 21521 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:38:49.328142 21521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:38:49.333842 21521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:44785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.144.126:45381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:49.335163 21521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:49.336309 21521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:49.346539 21529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:49.346554 21526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:49.346537 21527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:49.347011 21521 server_base.cc:1034] running on GCE node
I20260808 14:38:49.347405 21521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:49.348162 21521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:49.349332 21521 hybrid_clock.cc:648] HybridClock initialized: now 1786199929349310 us; error 34 us; skew 500 ppm
Aug 08 14:38:49 dist-test-slave-p3t5 krb5kdc[21115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199929, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:49.353487 21521 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:38:49.354980 21521 webserver.cc:460] Webserver started at http://127.19.144.67:38145/ using document root <none> and password file <none>
I20260808 14:38:49.355967 21521 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:49.356083 21521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:49.356405 21521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:49.360562 21521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "1a88c49dd60b4e0d885c784c910ef3bc"
format_stamp: "Formatted at 2026-08-08 14:38:49 on dist-test-slave-p3t5"
server_key: "e9248238fa4081123f929c7e35c67ebd"
server_key_iv: "432841649f40d75c8c0e523d89b3e852"
server_key_version: "encryptionkey@0"
I20260808 14:38:49.361387 21521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "1a88c49dd60b4e0d885c784c910ef3bc"
format_stamp: "Formatted at 2026-08-08 14:38:49 on dist-test-slave-p3t5"
server_key: "e9248238fa4081123f929c7e35c67ebd"
server_key_iv: "432841649f40d75c8c0e523d89b3e852"
server_key_version: "encryptionkey@0"
I20260808 14:38:49.365885 21521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.002s
I20260808 14:38:49.368744 21536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:49.370419 21521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:38:49.370572 21521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "1a88c49dd60b4e0d885c784c910ef3bc"
format_stamp: "Formatted at 2026-08-08 14:38:49 on dist-test-slave-p3t5"
server_key: "e9248238fa4081123f929c7e35c67ebd"
server_key_iv: "432841649f40d75c8c0e523d89b3e852"
server_key_version: "encryptionkey@0"
I20260808 14:38:49.370704 21521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:49.400426 21521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:49.404498 21521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:49.404728 21521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:49.406437 21521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:38:49.406519 21521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:49.406591 21521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:38:49.406630 21521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:49.418262 21521 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:40601
I20260808 14:38:49.419492 21521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
Aug 08 14:38:49 dist-test-slave-p3t5 krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199929, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:49.419934 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 21521
I20260808 14:38:49.420069 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:38:49.420387 20033 external_mini_cluster.cc:1428] Setting key c30ea812d06aab3815b8b6541fec5497
I20260808 14:38:49.425943 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.409722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:42355 (local address 127.19.144.126:45381)
0808 14:38:49.409883 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.409890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.409913 (+ 23us) server_negotiation.cc:407] Connection header received
0808 14:38:49.410888 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.410892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.410955 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.411066 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:49.413229 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.414276 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.416390 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.416924 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.417095 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.420935 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:49.420943 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:49.420950 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:49.420993 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:49.423360 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:49.423945 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:49.423952 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:49.423955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:49.424070 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:49.424433 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:49.424441 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:49.424444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:49.424759 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:49.424878 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.425452 (+ 574us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.425670 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 14:38:49.426748 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.408280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45381 (local address 127.19.144.67:42355)
0808 14:38:49.408980 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:49.409055 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:49.410592 (+ 1537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:49.411454 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:49.411483 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:49.412096 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:49.412989 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.413019 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.414505 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.414509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.416178 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.416195 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.417137 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.417142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.417370 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.418236 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:49.418258 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:49.420647 (+ 2389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:49.423527 (+ 2880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:49.423533 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:49.423547 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:49.423831 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:49.424179 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:49.424181 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:49.424183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:49.424325 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:49.424880 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:49.424886 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:49.425164 (+ 278us) client_negotiation.cc:769] Sending connection context
0808 14:38:49.425334 (+ 170us) client_negotiation.cc:240] Negotiation successful
0808 14:38:49.425608 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":171,"threads_started":1}
I20260808 14:38:49.426811 21653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:40601 every 8 connection(s)
I20260808 14:38:49.428208 21654 heartbeater.cc:347] Connected to a master server at 127.19.144.126:45381
I20260808 14:38:49.428395 21654 heartbeater.cc:464] Registering TS with master...
I20260808 14:38:49.428948 21654 heartbeater.cc:511] Master 127.19.144.126:45381 requested a full tablet report, sending...
I20260808 14:38:49.430176 21160 ts_manager.cc:194] Registered new tserver with Master: 1a88c49dd60b4e0d885c784c910ef3bc (127.19.144.67:40601)
I20260808 14:38:49.430815 21160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:42355
I20260808 14:38:49.434842 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:38:49.440304 21660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.431860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45381 (local address 127.19.144.67:53591)
0808 14:38:49.432190 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:49.432206 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:49.432316 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:49.432622 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:49.432625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:49.432865 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:49.432998 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.433007 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.433925 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.433929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.434765 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.434775 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.435354 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.435358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.435468 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.436086 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:49.436108 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:49.436456 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:49.438339 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:49.438344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:49.438349 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:49.438660 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:49.439019 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:49.439024 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:49.439027 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:49.439129 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:49.439723 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:49.439727 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:49.439850 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 14:38:49.439980 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 14:38:49.440143 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":131,"threads_started":1}
I20260808 14:38:49.440461 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.431750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:53591 (local address 127.19.144.126:45381)
0808 14:38:49.431907 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.431911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.432303 (+ 392us) server_negotiation.cc:407] Connection header received
0808 14:38:49.432410 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.432415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.432481 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.432590 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:49.433150 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.433802 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.434888 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.435236 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.435349 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.436609 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:49.436613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:49.436616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:49.436642 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:49.438216 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:49.438786 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:49.438791 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:49.438795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:49.438894 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:49.439249 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:49.439255 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:49.439258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:49.439564 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:49.439691 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.440068 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.440227 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 14:38:49 dist-test-slave-p3t5 krb5kdc[21115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199929, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 14:38:49 dist-test-slave-p3t5 krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199929, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:49.472929 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.461981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44514 (local address 127.19.144.126:45381)
0808 14:38:49.462126 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.462130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.462147 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:38:49.462196 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.462198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.462250 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.462331 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:49.463441 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.464007 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.465475 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.465776 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.465861 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.469272 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:49.469276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:49.469278 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:49.469307 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:49.471034 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:49.471605 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:49.471608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:49.471609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:49.471698 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:49.472032 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:49.472034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:49.472035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:49.472222 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:49.472322 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.472660 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.472772 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:38:49.476698 21160 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:38:49.479950 21160 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:38:49.497195 21675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.490571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:40601 (local address 127.0.0.1:41360)
0808 14:38:49.490857 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:49.490896 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:49.490994 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:49.492839 (+ 1845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:49.492842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:49.492860 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:49.492957 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.492964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.494419 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.494422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.495485 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.495495 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.496579 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.496583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.496722 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.496775 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 14:38:49.496866 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 14:38:49.497028 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":65,"thread_start_us":102,"threads_started":1}
I20260808 14:38:49.498198 21671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.489229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:35847 (local address 127.0.0.1:43780)
0808 14:38:49.489679 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:49.489740 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:49.489895 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:49.491988 (+ 2093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:49.491990 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:49.492012 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:49.492278 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.492288 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.494584 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.494587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.496604 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.496613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.497803 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.497806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.497899 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.497925 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 14:38:49.497977 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 14:38:49.498046 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":959,"spinlock_wait_cycles":384,"thread_start_us":121,"threads_started":1}
I20260808 14:38:49.498804 21673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.490163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:37901 (local address 127.0.0.1:50568)
0808 14:38:49.490430 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:49.490444 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:49.490532 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:49.491924 (+ 1392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:49.491930 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:49.491979 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:49.492311 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.492327 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.494153 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.494159 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.495994 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.496010 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.497924 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.497930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.498076 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.498116 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 14:38:49.498192 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 14:38:49.498568 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":88,"spinlock_wait_cycles":6784,"thread_start_us":112,"threads_started":1}
I20260808 14:38:49.498980 21676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.490792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41360 (local address 127.19.144.67:40601)
0808 14:38:49.492390 (+ 1598us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.492406 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.492429 (+ 23us) server_negotiation.cc:407] Connection header received
0808 14:38:49.492502 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.492509 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.492684 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.493100 (+ 416us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:49.493228 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.494271 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.495615 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.496444 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.498470 (+ 2026us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.498661 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.498724 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.498831 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1458,"thread_start_us":170,"threads_started":1}
I20260808 14:38:49.499053 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.489344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43780 (local address 127.19.144.66:35847)
0808 14:38:49.491433 (+ 2089us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.491449 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.491476 (+ 27us) server_negotiation.cc:407] Connection header received
0808 14:38:49.491589 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.491598 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.491834 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.492011 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:49.492430 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.493655 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.496739 (+ 3084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.497673 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.498484 (+ 811us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.498772 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.498823 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.498906 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1935,"thread_start_us":150,"threads_started":1}
I20260808 14:38:49.501863 21437 tablet_service.cc:1467] Processing CreateTablet for tablet 10efea53c7164c109e60c3f2babfa23c (DEFAULT_TABLE table=test-table [id=2f25cb85385f43c6b2cd14587fd45d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:49.502249 21587 tablet_service.cc:1467] Processing CreateTablet for tablet 10efea53c7164c109e60c3f2babfa23c (DEFAULT_TABLE table=test-table [id=2f25cb85385f43c6b2cd14587fd45d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:49.503104 21437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10efea53c7164c109e60c3f2babfa23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:49.503517 21587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10efea53c7164c109e60c3f2babfa23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:49.506229 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.490308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50568 (local address 127.19.144.65:37901)
0808 14:38:49.491235 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.491243 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.491267 (+ 24us) server_negotiation.cc:407] Connection header received
0808 14:38:49.491348 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.491354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.491603 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.491759 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:49.492481 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.493973 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.496171 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.497229 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.497708 (+ 479us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.499413 (+ 1705us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.499761 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.506029 (+ 6268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":813,"thread_start_us":94,"threads_started":1}
I20260808 14:38:49.510555 21301 tablet_service.cc:1467] Processing CreateTablet for tablet 10efea53c7164c109e60c3f2babfa23c (DEFAULT_TABLE table=test-table [id=2f25cb85385f43c6b2cd14587fd45d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:49.511735 21301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10efea53c7164c109e60c3f2babfa23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:49.514676 21677 tablet_bootstrap.cc:492] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe: Bootstrap starting.
I20260808 14:38:49.517171 21678 tablet_bootstrap.cc:492] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc: Bootstrap starting.
I20260808 14:38:49.517714 21677 tablet_bootstrap.cc:654] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:49.518851 21677 log.cc:824] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe: Log is configured to *not* fsync() on all Append() calls
I20260808 14:38:49.520807 21679 tablet_bootstrap.cc:492] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b: Bootstrap starting.
I20260808 14:38:49.521030 21678 tablet_bootstrap.cc:654] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:49.521428 21677 tablet_bootstrap.cc:492] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe: No bootstrap required, opened a new log
I20260808 14:38:49.521745 21677 ts_tablet_manager.cc:1397] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.005s
I20260808 14:38:49.522161 21678 log.cc:824] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc: Log is configured to *not* fsync() on all Append() calls
I20260808 14:38:49.523147 21679 tablet_bootstrap.cc:654] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:49.524442 21678 tablet_bootstrap.cc:492] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc: No bootstrap required, opened a new log
I20260808 14:38:49.524446 21679 log.cc:824] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b: Log is configured to *not* fsync() on all Append() calls
I20260808 14:38:49.524626 21678 ts_tablet_manager.cc:1397] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260808 14:38:49.526600 21679 tablet_bootstrap.cc:492] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b: No bootstrap required, opened a new log
I20260808 14:38:49.526818 21679 ts_tablet_manager.cc:1397] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:38:49.527498 21677 raft_consensus.cc:348] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.527835 21677 raft_consensus.cc:374] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:49.527892 21677 raft_consensus.cc:729] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d870ceb7bea47058f8be6b9ba840efe, State: Initialized, Role: FOLLOWER
I20260808 14:38:49.528475 21677 consensus_queue.cc:260] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.529829 21677 ts_tablet_manager.cc:1428] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260808 14:38:49.530256 21511 heartbeater.cc:503] Master 127.19.144.126:45381 was elected leader, sending a full tablet report...
I20260808 14:38:49.530020 21678 raft_consensus.cc:348] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.531540 21678 raft_consensus.cc:374] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:49.531620 21678 raft_consensus.cc:729] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a88c49dd60b4e0d885c784c910ef3bc, State: Initialized, Role: FOLLOWER
I20260808 14:38:49.531381 21679 raft_consensus.cc:348] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.531940 21679 raft_consensus.cc:374] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:49.532035 21679 raft_consensus.cc:729] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d1e42ebd8534d948044e34753d7553b, State: Initialized, Role: FOLLOWER
I20260808 14:38:49.532151 21678 consensus_queue.cc:260] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.532567 21679 consensus_queue.cc:260] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.533227 21678 ts_tablet_manager.cc:1428] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260808 14:38:49.533667 21679 ts_tablet_manager.cc:1428] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 14:38:49.533598 21654 heartbeater.cc:503] Master 127.19.144.126:45381 was elected leader, sending a full tablet report...
I20260808 14:38:49.534116 21368 heartbeater.cc:503] Master 127.19.144.126:45381 was elected leader, sending a full tablet report...
W20260808 14:38:49.664893 21512 tablet.cc:2369] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:38:49.665942 21684 raft_consensus.cc:482] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:38:49.666178 21684 raft_consensus.cc:504] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.667523 21684 leader_election.cc:288] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d870ceb7bea47058f8be6b9ba840efe (127.19.144.66:35847), 7d1e42ebd8534d948044e34753d7553b (127.19.144.65:37901)
W20260808 14:38:49.671407 21655 tablet.cc:2369] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:38:49.672449 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.667809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:35847 (local address 127.19.144.67:53693)
0808 14:38:49.667984 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:49.668023 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:49.668152 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:49.668576 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:49.668579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:49.668600 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:49.668757 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.668765 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.669790 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.669794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.671062 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.671073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.671934 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.671938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.672070 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.672103 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 14:38:49.672203 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 14:38:49.672271 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":1}
I20260808 14:38:49.672458 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.667964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:53693 (local address 127.19.144.66:35847)
0808 14:38:49.668118 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.668123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.668142 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:38:49.668338 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.668343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.668398 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.668514 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:49.668889 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.669660 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.671193 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.671782 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.671915 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.671993 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.672229 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.672311 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:38:49.673360 21465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10efea53c7164c109e60c3f2babfa23c" candidate_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d870ceb7bea47058f8be6b9ba840efe" is_pre_election: true
I20260808 14:38:49.673465 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.668185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:37901 (local address 127.19.144.67:43179)
0808 14:38:49.668600 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:49.668615 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:49.668709 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:49.668984 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:49.668987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:49.669006 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:49.669135 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.669141 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.670560 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.670562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.672266 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.672276 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.673141 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.673143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.673216 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.673235 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 14:38:49.673297 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 14:38:49.673350 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":125,"threads_started":1}
I20260808 14:38:49.673573 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.668333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:43179 (local address 127.19.144.65:37901)
0808 14:38:49.668489 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.668496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.668714 (+ 218us) server_negotiation.cc:407] Connection header received
0808 14:38:49.668781 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.668784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.668850 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.668940 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:49.669503 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.670414 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.672410 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.672988 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.673130 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.673182 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.673343 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.673438 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:38:49.673810 21465 raft_consensus.cc:2454] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a88c49dd60b4e0d885c784c910ef3bc in term 0.
I20260808 14:38:49.674340 21321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10efea53c7164c109e60c3f2babfa23c" candidate_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d1e42ebd8534d948044e34753d7553b" is_pre_election: true
I20260808 14:38:49.674450 21537 leader_election.cc:302] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a88c49dd60b4e0d885c784c910ef3bc, 4d870ceb7bea47058f8be6b9ba840efe; no voters:
I20260808 14:38:49.674680 21321 raft_consensus.cc:2454] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a88c49dd60b4e0d885c784c910ef3bc in term 0.
I20260808 14:38:49.674885 21684 raft_consensus.cc:2781] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:38:49.675004 21684 raft_consensus.cc:482] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:38:49.675071 21684 raft_consensus.cc:3037] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:49.676330 21684 raft_consensus.cc:504] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.676755 21684 leader_election.cc:288] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [CANDIDATE]: Term 1 election: Requested vote from peers 4d870ceb7bea47058f8be6b9ba840efe (127.19.144.66:35847), 7d1e42ebd8534d948044e34753d7553b (127.19.144.65:37901)
I20260808 14:38:49.677175 21465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10efea53c7164c109e60c3f2babfa23c" candidate_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d870ceb7bea47058f8be6b9ba840efe"
I20260808 14:38:49.677186 21321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10efea53c7164c109e60c3f2babfa23c" candidate_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d1e42ebd8534d948044e34753d7553b"
I20260808 14:38:49.677330 21321 raft_consensus.cc:3037] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:49.677302 21465 raft_consensus.cc:3037] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:49.678578 21321 raft_consensus.cc:2454] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a88c49dd60b4e0d885c784c910ef3bc in term 1.
I20260808 14:38:49.678654 21465 raft_consensus.cc:2454] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a88c49dd60b4e0d885c784c910ef3bc in term 1.
I20260808 14:38:49.679028 21537 leader_election.cc:302] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a88c49dd60b4e0d885c784c910ef3bc, 4d870ceb7bea47058f8be6b9ba840efe; no voters:
I20260808 14:38:49.679433 21684 raft_consensus.cc:2781] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:38:49.679731 21684 raft_consensus.cc:686] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [term 1 LEADER]: Becoming Leader. State: Replica: 1a88c49dd60b4e0d885c784c910ef3bc, State: Running, Role: LEADER
I20260808 14:38:49.680135 21684 consensus_queue.cc:237] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.683884 21160 catalog_manager.cc:5495] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a88c49dd60b4e0d885c784c910ef3bc (127.19.144.67). New cstate: current_term: 1 leader_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } health_report { overall_health: HEALTHY } } }
W20260808 14:38:49.685055 21369 tablet.cc:2369] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:38:49.739403 21676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.735450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41374 (local address 127.19.144.67:40601)
0808 14:38:49.735612 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.735617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.735633 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:38:49.735880 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.735883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.735954 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.736061 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:38:49.736354 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.736876 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.737814 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.738062 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.738177 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.738525 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:38:49.738964 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:38:49.739097 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.739190 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.739250 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 14:38:49.746016 21465 raft_consensus.cc:1261] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe [term 1 FOLLOWER]: Refusing update from remote peer 1a88c49dd60b4e0d885c784c910ef3bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:38:49.746047 21321 raft_consensus.cc:1261] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b [term 1 FOLLOWER]: Refusing update from remote peer 1a88c49dd60b4e0d885c784c910ef3bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:38:49.746771 21684 consensus_queue.cc:1035] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:38:49.747021 21687 consensus_queue.cc:1035] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:38:49.757993 21691 mvcc.cc:205] Tried to move back new op lower bound from 7316274912231870464 to 7316274911972945920. Current Snapshot: MvccSnapshot[applied={T|T < 7316274912231870464 or (T in {7316274912231870464})}]
I20260808 14:38:49.767253 21162 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:38:49.767453 21162 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:38:49.770169 21162 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f: Sending DeleteTablet for 3 replicas of tablet 10efea53c7164c109e60c3f2babfa23c
I20260808 14:38:49.771129 21437 tablet_service.cc:1514] Processing DeleteTablet for tablet 10efea53c7164c109e60c3f2babfa23c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:49 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:43780
I20260808 14:38:49.771139 21587 tablet_service.cc:1514] Processing DeleteTablet for tablet 10efea53c7164c109e60c3f2babfa23c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:49 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:41360
I20260808 14:38:49.771409 21301 tablet_service.cc:1514] Processing DeleteTablet for tablet 10efea53c7164c109e60c3f2babfa23c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:49 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:50568
I20260808 14:38:49.771859 21702 tablet_replica.cc:330] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc: stopping tablet replica
I20260808 14:38:49.771916 21700 tablet_replica.cc:330] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe: stopping tablet replica
I20260808 14:38:49.772177 21162 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 14:38:49.772275 21700 raft_consensus.cc:2229] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:38:49.772275 21702 raft_consensus.cc:2229] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:38:49.772636 21700 raft_consensus.cc:2258] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 14:38:49.772640 21162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:38:49.773039 21702 raft_consensus.cc:2258] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:38:49.773190 21703 tablet_replica.cc:330] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b: stopping tablet replica
I20260808 14:38:49.773525 21703 raft_consensus.cc:2229] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:38:49.773890 21703 raft_consensus.cc:2258] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:38:49.774348 21700 ts_tablet_manager.cc:1905] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:49.775635 21703 ts_tablet_manager.cc:1905] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:49.777462 21700 ts_tablet_manager.cc:1918] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:38:49.777582 21700 log.cc:1196] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/10efea53c7164c109e60c3f2babfa23c
I20260808 14:38:49.777980 21700 ts_tablet_manager.cc:1939] T 10efea53c7164c109e60c3f2babfa23c P 4d870ceb7bea47058f8be6b9ba840efe: Deleting consensus metadata
I20260808 14:38:49.779093 21703 ts_tablet_manager.cc:1918] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:38:49.779273 21703 log.cc:1196] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/wal/wals/10efea53c7164c109e60c3f2babfa23c
I20260808 14:38:49.779510 21587 tablet_service.cc:1467] Processing CreateTablet for tablet acdb8d196a7943cb9012e4f8ea500060 (DEFAULT_TABLE table=test-table [id=e6372e3dad6d44a48142fc1deb958f73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:49.779582 21144 catalog_manager.cc:4841] TS 4d870ceb7bea47058f8be6b9ba840efe (127.19.144.66:35847): tablet 10efea53c7164c109e60c3f2babfa23c (table test-table [id=2f25cb85385f43c6b2cd14587fd45d1b]) successfully deleted
I20260808 14:38:49.779255 21301 tablet_service.cc:1467] Processing CreateTablet for tablet acdb8d196a7943cb9012e4f8ea500060 (DEFAULT_TABLE table=test-table [id=e6372e3dad6d44a48142fc1deb958f73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:49.779816 21587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acdb8d196a7943cb9012e4f8ea500060. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:49.779980 21301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acdb8d196a7943cb9012e4f8ea500060. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:49.779982 21703 ts_tablet_manager.cc:1939] T 10efea53c7164c109e60c3f2babfa23c P 7d1e42ebd8534d948044e34753d7553b: Deleting consensus metadata
I20260808 14:38:49.781949 21437 tablet_service.cc:1467] Processing CreateTablet for tablet acdb8d196a7943cb9012e4f8ea500060 (DEFAULT_TABLE table=test-table [id=e6372e3dad6d44a48142fc1deb958f73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:49.782217 21437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acdb8d196a7943cb9012e4f8ea500060. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:49.782279 21679 tablet_bootstrap.cc:492] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b: Bootstrap starting.
I20260808 14:38:49.782279 21678 tablet_bootstrap.cc:492] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc: Bootstrap starting.
I20260808 14:38:49.783317 21678 tablet_bootstrap.cc:654] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:49.783587 21679 tablet_bootstrap.cc:654] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:49.784854 21702 ts_tablet_manager.cc:1905] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:49.784929 21677 tablet_bootstrap.cc:492] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe: Bootstrap starting.
I20260808 14:38:49.785370 21144 catalog_manager.cc:4841] TS 7d1e42ebd8534d948044e34753d7553b (127.19.144.65:37901): tablet 10efea53c7164c109e60c3f2babfa23c (table test-table [id=2f25cb85385f43c6b2cd14587fd45d1b]) successfully deleted
I20260808 14:38:49.785971 21677 tablet_bootstrap.cc:654] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:49.787047 21679 tablet_bootstrap.cc:492] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b: No bootstrap required, opened a new log
I20260808 14:38:49.787148 21679 ts_tablet_manager.cc:1397] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 14:38:49.787712 21702 ts_tablet_manager.cc:1918] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:38:49.787748 21679 raft_consensus.cc:348] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.787842 21702 log.cc:1196] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/10efea53c7164c109e60c3f2babfa23c
I20260808 14:38:49.787889 21679 raft_consensus.cc:374] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:49.787946 21679 raft_consensus.cc:729] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d1e42ebd8534d948044e34753d7553b, State: Initialized, Role: FOLLOWER
I20260808 14:38:49.788062 21677 tablet_bootstrap.cc:492] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe: No bootstrap required, opened a new log
I20260808 14:38:49.788151 21677 ts_tablet_manager.cc:1397] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:38:49.788097 21679 consensus_queue.cc:260] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.788322 21702 ts_tablet_manager.cc:1939] T 10efea53c7164c109e60c3f2babfa23c P 1a88c49dd60b4e0d885c784c910ef3bc: Deleting consensus metadata
I20260808 14:38:49.788375 21679 ts_tablet_manager.cc:1428] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 14:38:49.788617 21677 raft_consensus.cc:348] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.788748 21677 raft_consensus.cc:374] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:49.788796 21677 raft_consensus.cc:729] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d870ceb7bea47058f8be6b9ba840efe, State: Initialized, Role: FOLLOWER
I20260808 14:38:49.788924 21677 consensus_queue.cc:260] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.789186 21677 ts_tablet_manager.cc:1428] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:38:49.789772 21145 catalog_manager.cc:4841] TS 1a88c49dd60b4e0d885c784c910ef3bc (127.19.144.67:40601): tablet 10efea53c7164c109e60c3f2babfa23c (table test-table [id=2f25cb85385f43c6b2cd14587fd45d1b]) successfully deleted
I20260808 14:38:49.790771 21678 tablet_bootstrap.cc:492] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc: No bootstrap required, opened a new log
I20260808 14:38:49.790874 21678 ts_tablet_manager.cc:1397] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260808 14:38:49.791329 21678 raft_consensus.cc:348] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.791453 21678 raft_consensus.cc:374] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:49.791486 21678 raft_consensus.cc:729] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a88c49dd60b4e0d885c784c910ef3bc, State: Initialized, Role: FOLLOWER
I20260808 14:38:49.791603 21678 consensus_queue.cc:260] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.791842 21678 ts_tablet_manager.cc:1428] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:38:49.819963 21683 raft_consensus.cc:482] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:38:49.820143 21683 raft_consensus.cc:504] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.821494 21683 leader_election.cc:288] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d1e42ebd8534d948044e34753d7553b (127.19.144.65:37901), 1a88c49dd60b4e0d885c784c910ef3bc (127.19.144.67:40601)
I20260808 14:38:49.825908 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.821733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:37901 (local address 127.19.144.66:39281)
0808 14:38:49.822126 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:49.822141 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:49.822247 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:49.822511 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:49.822513 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:49.822527 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:49.822702 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.822710 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.823821 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.823824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.824691 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.824701 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.825425 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.825429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.825535 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.825562 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 14:38:49.825661 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 14:38:49.825735 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":21,"thread_start_us":111,"threads_started":1}
I20260808 14:38:49.825911 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.822108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:39281 (local address 127.19.144.65:37901)
0808 14:38:49.822239 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.822243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.822259 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:38:49.822347 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.822350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.822408 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.822495 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:49.822838 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.823663 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.824820 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.825276 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.825401 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.825466 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.825692 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.825773 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 14:38:49.826524 21707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.821786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:40601 (local address 127.19.144.66:43153)
0808 14:38:49.822229 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:49.822242 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:49.822367 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:49.822636 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:49.822640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:49.822659 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:49.822824 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.822831 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.823791 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.823795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.825149 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.825161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.826113 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.826118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.826215 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.826232 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 14:38:49.826320 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 14:38:49.826386 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":76,"thread_start_us":87,"threads_started":1}
I20260808 14:38:49.826536 21676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.821849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:43153 (local address 127.19.144.67:40601)
0808 14:38:49.822066 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.822071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.822309 (+ 238us) server_negotiation.cc:407] Connection header received
0808 14:38:49.822493 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.822497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.822545 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.822614 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:49.822941 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.823663 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.825295 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.825852 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.825983 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.826042 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.826351 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.826422 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260808 14:38:49.827196 21321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acdb8d196a7943cb9012e4f8ea500060" candidate_uuid: "4d870ceb7bea47058f8be6b9ba840efe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d1e42ebd8534d948044e34753d7553b" is_pre_election: true
I20260808 14:38:49.827183 21607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acdb8d196a7943cb9012e4f8ea500060" candidate_uuid: "4d870ceb7bea47058f8be6b9ba840efe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" is_pre_election: true
I20260808 14:38:49.827355 21321 raft_consensus.cc:2454] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d870ceb7bea47058f8be6b9ba840efe in term 0.
I20260808 14:38:49.827416 21607 raft_consensus.cc:2454] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d870ceb7bea47058f8be6b9ba840efe in term 0.
I20260808 14:38:49.827772 21394 leader_election.cc:302] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1a88c49dd60b4e0d885c784c910ef3bc, 4d870ceb7bea47058f8be6b9ba840efe, 7d1e42ebd8534d948044e34753d7553b; no voters:
I20260808 14:38:49.828080 21683 raft_consensus.cc:2781] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:38:49.828162 21683 raft_consensus.cc:482] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:38:49.828209 21683 raft_consensus.cc:3037] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:49.829487 21683 raft_consensus.cc:504] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.829921 21683 leader_election.cc:288] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [CANDIDATE]: Term 1 election: Requested vote from peers 7d1e42ebd8534d948044e34753d7553b (127.19.144.65:37901), 1a88c49dd60b4e0d885c784c910ef3bc (127.19.144.67:40601)
I20260808 14:38:49.830287 21321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acdb8d196a7943cb9012e4f8ea500060" candidate_uuid: "4d870ceb7bea47058f8be6b9ba840efe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d1e42ebd8534d948044e34753d7553b"
I20260808 14:38:49.830291 21607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acdb8d196a7943cb9012e4f8ea500060" candidate_uuid: "4d870ceb7bea47058f8be6b9ba840efe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a88c49dd60b4e0d885c784c910ef3bc"
I20260808 14:38:49.830426 21321 raft_consensus.cc:3037] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:49.830432 21607 raft_consensus.cc:3037] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:49.831343 21321 raft_consensus.cc:2454] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d870ceb7bea47058f8be6b9ba840efe in term 1.
I20260808 14:38:49.831681 21394 leader_election.cc:302] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d870ceb7bea47058f8be6b9ba840efe, 7d1e42ebd8534d948044e34753d7553b; no voters:
I20260808 14:38:49.831879 21683 raft_consensus.cc:2781] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:38:49.831916 21607 raft_consensus.cc:2454] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d870ceb7bea47058f8be6b9ba840efe in term 1.
I20260808 14:38:49.832118 21683 raft_consensus.cc:686] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [term 1 LEADER]: Becoming Leader. State: Replica: 4d870ceb7bea47058f8be6b9ba840efe, State: Running, Role: LEADER
I20260808 14:38:49.832376 21683 consensus_queue.cc:237] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.834808 21162 catalog_manager.cc:5495] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d870ceb7bea47058f8be6b9ba840efe (127.19.144.66). New cstate: current_term: 1 leader_uuid: "4d870ceb7bea47058f8be6b9ba840efe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } health_report { overall_health: UNKNOWN } } }
Aug 08 14:38:49 dist-test-slave-p3t5 krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199928, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:49.881182 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.871670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44518 (local address 127.19.144.126:45381)
0808 14:38:49.871822 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.871827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.871987 (+ 160us) server_negotiation.cc:407] Connection header received
0808 14:38:49.872109 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.872115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.872167 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.872268 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:49.872991 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.873522 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.874521 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.874735 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.874839 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.877486 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:49.877490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:49.877491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:49.877512 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:49.879107 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:49.879725 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:49.879729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:49.879730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:49.879827 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:49.880154 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:49.880157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:49.880158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:49.880343 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:49.880420 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.880807 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.880995 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":64}
I20260808 14:38:49.881194 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.871564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45381 (local address 127.0.0.1:44518)
0808 14:38:49.871869 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:49.871886 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:49.872012 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:49.872319 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:49.872321 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:49.872722 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:49.872823 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.872829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.873658 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.873664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.874396 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.874407 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.874856 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.874860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.874934 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.875625 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:49.875637 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:49.877328 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:49.879231 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:49.879242 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:49.879250 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:49.879592 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:49.879935 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:49.879938 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:49.879940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:49.880049 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:49.880453 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:49.880461 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:49.880618 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 14:38:49.880743 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 14:38:49.880918 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":131,"threads_started":1}
I20260808 14:38:49.885969 21162 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:44518:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 14:38:49.886833 21162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:38:49.892149 21301 tablet_service.cc:1467] Processing CreateTablet for tablet 2bc870957da244218e51daa1cfc00852 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1d05763427e4e19bdf0e36df1e8ddb7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:38:49.892204 21437 tablet_service.cc:1467] Processing CreateTablet for tablet 2bc870957da244218e51daa1cfc00852 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1d05763427e4e19bdf0e36df1e8ddb7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:38:49.892643 21301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bc870957da244218e51daa1cfc00852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:49.892684 21437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bc870957da244218e51daa1cfc00852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:49.895747 21679 tablet_bootstrap.cc:492] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b: Bootstrap starting.
I20260808 14:38:49.896047 21677 tablet_bootstrap.cc:492] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe: Bootstrap starting.
I20260808 14:38:49.895978 21587 tablet_service.cc:1467] Processing CreateTablet for tablet 2bc870957da244218e51daa1cfc00852 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1d05763427e4e19bdf0e36df1e8ddb7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:38:49.896364 21587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bc870957da244218e51daa1cfc00852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:49.896795 21679 tablet_bootstrap.cc:654] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:49.897055 21677 tablet_bootstrap.cc:654] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:49.898326 21679 tablet_bootstrap.cc:492] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b: No bootstrap required, opened a new log
I20260808 14:38:49.898403 21679 ts_tablet_manager.cc:1397] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:38:49.898984 21679 raft_consensus.cc:348] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.899103 21679 raft_consensus.cc:374] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:49.899135 21679 raft_consensus.cc:729] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d1e42ebd8534d948044e34753d7553b, State: Initialized, Role: FOLLOWER
I20260808 14:38:49.899210 21677 tablet_bootstrap.cc:492] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe: No bootstrap required, opened a new log
I20260808 14:38:49.899250 21679 consensus_queue.cc:260] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.899295 21677 ts_tablet_manager.cc:1397] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:38:49.899549 21679 ts_tablet_manager.cc:1428] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:38:49.899747 21677 raft_consensus.cc:348] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.899871 21677 raft_consensus.cc:374] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:49.899905 21677 raft_consensus.cc:729] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d870ceb7bea47058f8be6b9ba840efe, State: Initialized, Role: FOLLOWER
I20260808 14:38:49.899911 21678 tablet_bootstrap.cc:492] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc: Bootstrap starting.
I20260808 14:38:49.899866 21685 tablet_replica.cc:439] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } } }
I20260808 14:38:49.900022 21685 tablet_replica.cc:442] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:38:49.900023 21677 consensus_queue.cc:260] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.900290 21677 ts_tablet_manager.cc:1428] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 14:38:49.900924 21678 tablet_bootstrap.cc:654] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:49.902004 21709 tablet_replica.cc:439] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } } }
I20260808 14:38:49.902163 21709 tablet_replica.cc:442] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:38:49.903560 21678 tablet_bootstrap.cc:492] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc: No bootstrap required, opened a new log
I20260808 14:38:49.903678 21678 ts_tablet_manager.cc:1397] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260808 14:38:49.904081 21678 raft_consensus.cc:348] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.904176 21678 raft_consensus.cc:374] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:49.904199 21678 raft_consensus.cc:729] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a88c49dd60b4e0d885c784c910ef3bc, State: Initialized, Role: FOLLOWER
I20260808 14:38:49.904340 21678 consensus_queue.cc:260] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.904626 21678 ts_tablet_manager.cc:1428] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 14:38:49.904577 21687 tablet_replica.cc:439] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } } }
I20260808 14:38:49.904702 21687 tablet_replica.cc:442] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:38:49.930907 21685 raft_consensus.cc:482] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:38:49.931103 21685 raft_consensus.cc:504] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.932490 21685 leader_election.cc:288] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d870ceb7bea47058f8be6b9ba840efe (127.19.144.66:35847), 1a88c49dd60b4e0d885c784c910ef3bc (127.19.144.67:40601)
I20260808 14:38:49.938273 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.933149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:43441 (local address 127.19.144.66:35847)
0808 14:38:49.933410 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.933416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.933432 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:38:49.933577 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.933581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.933688 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.933738 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:49.934181 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.934965 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.936889 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.937497 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.937596 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.937695 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.938039 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.938124 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173}
I20260808 14:38:49.938483 21676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.933275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:54455 (local address 127.19.144.67:40601)
0808 14:38:49.933522 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.933526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.933975 (+ 449us) server_negotiation.cc:407] Connection header received
0808 14:38:49.934158 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.934162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.934206 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.934267 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:49.934724 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.935493 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.937102 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.937772 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.937866 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.937915 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.938256 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.938335 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260808 14:38:49.938715 21720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.932739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:35847 (local address 127.19.144.65:43441)
0808 14:38:49.933286 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:49.933305 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:49.933411 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:49.933905 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:49.933907 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:49.933927 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:49.934049 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.934055 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.935132 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.935135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.936741 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.936753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.937796 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.937799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.937894 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.937917 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 14:38:49.938487 (+ 570us) client_negotiation.cc:240] Negotiation successful
0808 14:38:49.938568 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":38,"thread_start_us":122,"threads_started":1}
I20260808 14:38:49.939155 21465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc870957da244218e51daa1cfc00852" candidate_uuid: "7d1e42ebd8534d948044e34753d7553b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d870ceb7bea47058f8be6b9ba840efe" is_pre_election: true
I20260808 14:38:49.939314 21465 raft_consensus.cc:2454] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d1e42ebd8534d948044e34753d7553b in term 0.
I20260808 14:38:49.939473 21721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.932823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:40601 (local address 127.19.144.65:54455)
0808 14:38:49.933882 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:49.933897 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:49.934044 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:49.934447 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:49.934450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:49.934467 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:49.934589 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.934594 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.935699 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.935702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.936955 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.936965 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.938009 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.938012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.938130 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.938151 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 14:38:49.939124 (+ 973us) client_negotiation.cc:240] Negotiation successful
0808 14:38:49.939207 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":971,"mutex_wait_us":1,"thread_start_us":124,"threads_started":1}
I20260808 14:38:49.939714 21251 leader_election.cc:302] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d870ceb7bea47058f8be6b9ba840efe, 7d1e42ebd8534d948044e34753d7553b; no voters:
I20260808 14:38:49.940343 21685 raft_consensus.cc:2781] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:38:49.940407 21685 raft_consensus.cc:482] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:38:49.940459 21685 raft_consensus.cc:3037] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:49.940608 21607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc870957da244218e51daa1cfc00852" candidate_uuid: "7d1e42ebd8534d948044e34753d7553b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" is_pre_election: true
I20260808 14:38:49.940768 21607 raft_consensus.cc:2454] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d1e42ebd8534d948044e34753d7553b in term 0.
I20260808 14:38:49.941701 21685 raft_consensus.cc:504] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.942122 21685 leader_election.cc:288] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [CANDIDATE]: Term 1 election: Requested vote from peers 4d870ceb7bea47058f8be6b9ba840efe (127.19.144.66:35847), 1a88c49dd60b4e0d885c784c910ef3bc (127.19.144.67:40601)
I20260808 14:38:49.942541 21465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc870957da244218e51daa1cfc00852" candidate_uuid: "7d1e42ebd8534d948044e34753d7553b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d870ceb7bea47058f8be6b9ba840efe"
I20260808 14:38:49.942674 21465 raft_consensus.cc:3037] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:49.942734 21607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc870957da244218e51daa1cfc00852" candidate_uuid: "7d1e42ebd8534d948044e34753d7553b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a88c49dd60b4e0d885c784c910ef3bc"
I20260808 14:38:49.942842 21607 raft_consensus.cc:3037] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:49.943853 21465 raft_consensus.cc:2454] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d1e42ebd8534d948044e34753d7553b in term 1.
I20260808 14:38:49.943948 21607 raft_consensus.cc:2454] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d1e42ebd8534d948044e34753d7553b in term 1.
I20260808 14:38:49.944356 21252 leader_election.cc:302] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a88c49dd60b4e0d885c784c910ef3bc, 7d1e42ebd8534d948044e34753d7553b; no voters:
I20260808 14:38:49.944633 21685 raft_consensus.cc:2781] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:38:49.944844 21685 raft_consensus.cc:686] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [term 1 LEADER]: Becoming Leader. State: Replica: 7d1e42ebd8534d948044e34753d7553b, State: Running, Role: LEADER
I20260808 14:38:49.945097 21685 consensus_queue.cc:237] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } }
I20260808 14:38:49.947285 21722 tablet_replica.cc:439] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b: TxnStatusTablet state changed. Reason: New leader 7d1e42ebd8534d948044e34753d7553b. Latest consensus state: current_term: 1 leader_uuid: "7d1e42ebd8534d948044e34753d7553b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } } }
I20260808 14:38:49.947419 21722 tablet_replica.cc:442] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:38:49.947711 21724 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 14:38:49.947932 21724 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 14:38:49.947855 21162 catalog_manager.cc:5495] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d1e42ebd8534d948044e34753d7553b (127.19.144.65). New cstate: current_term: 1 leader_uuid: "7d1e42ebd8534d948044e34753d7553b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } health_report { overall_health: UNKNOWN } } }
I20260808 14:38:49.962473 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.957963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50580 (local address 127.19.144.65:37901)
0808 14:38:49.958092 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.958103 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.958219 (+ 116us) server_negotiation.cc:407] Connection header received
0808 14:38:49.958425 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.958428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.958487 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.958573 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:38:49.958880 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.959598 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.960872 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.961108 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.961194 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.961560 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:38:49.962068 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:38:49.962158 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.962292 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.962352 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:38:49.962533 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.957861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:37901 (local address 127.0.0.1:50580)
0808 14:38:49.958137 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:49.958154 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:49.958282 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:49.958625 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:49.958629 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:49.958638 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:38:49.958762 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.958769 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.959718 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.959722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.960749 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.960761 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.961233 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.961237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.961338 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.961409 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:38:49.962168 (+ 759us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:38:49.962189 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 14:38:49.962274 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 14:38:49.962374 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260808 14:38:49.966084 21465 raft_consensus.cc:1261] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe [term 1 FOLLOWER]: Refusing update from remote peer 7d1e42ebd8534d948044e34753d7553b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:38:49.966176 21607 raft_consensus.cc:1261] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc [term 1 FOLLOWER]: Refusing update from remote peer 7d1e42ebd8534d948044e34753d7553b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:38:49.966689 21722 consensus_queue.cc:1035] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:38:49.966925 21685 consensus_queue.cc:1035] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:38:49.968581 21709 tablet_replica.cc:439] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe: TxnStatusTablet state changed. Reason: New leader 7d1e42ebd8534d948044e34753d7553b. Latest consensus state: current_term: 1 leader_uuid: "7d1e42ebd8534d948044e34753d7553b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } } }
I20260808 14:38:49.968706 21709 tablet_replica.cc:442] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:38:49.970340 21687 tablet_replica.cc:439] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc: TxnStatusTablet state changed. Reason: New leader 7d1e42ebd8534d948044e34753d7553b. Latest consensus state: current_term: 1 leader_uuid: "7d1e42ebd8534d948044e34753d7553b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } } }
I20260808 14:38:49.970484 21687 tablet_replica.cc:442] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:38:49.971952 21685 tablet_replica.cc:439] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7d1e42ebd8534d948044e34753d7553b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } } }
I20260808 14:38:49.972075 21685 tablet_replica.cc:442] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:38:49.972841 21692 mvcc.cc:205] Tried to move back new op lower bound from 7316274913133834240 to 7316274913057898496. Current Snapshot: MvccSnapshot[applied={T|T < 7316274913133834240 or (T in {7316274913133834240})}]
I20260808 14:38:49.974380 21722 tablet_replica.cc:439] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7d1e42ebd8534d948044e34753d7553b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } } }
I20260808 14:38:49.974406 21709 tablet_replica.cc:439] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7d1e42ebd8534d948044e34753d7553b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } } }
I20260808 14:38:49.974522 21709 tablet_replica.cc:442] T 2bc870957da244218e51daa1cfc00852 P 4d870ceb7bea47058f8be6b9ba840efe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:38:49.974522 21722 tablet_replica.cc:442] T 2bc870957da244218e51daa1cfc00852 P 7d1e42ebd8534d948044e34753d7553b: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:38:49.976490 21690 mvcc.cc:205] Tried to move back new op lower bound from 7316274913133834240 to 7316274913057898496. Current Snapshot: MvccSnapshot[applied={T|T < 7316274913133834240}]
I20260808 14:38:49.976680 21687 tablet_replica.cc:439] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7d1e42ebd8534d948044e34753d7553b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 } } peers { permanent_uuid: "4d870ceb7bea47058f8be6b9ba840efe" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35847 } } peers { permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 } } }
I20260808 14:38:49.976814 21687 tablet_replica.cc:442] T 2bc870957da244218e51daa1cfc00852 P 1a88c49dd60b4e0d885c784c910ef3bc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:38:49.983616 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.979622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43792 (local address 127.19.144.66:35847)
0808 14:38:49.979739 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.979743 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.979756 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:38:49.979931 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.979934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.979988 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.980084 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:38:49.980359 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.980853 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.982113 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.982329 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.982413 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.982712 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:38:49.983196 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:38:49.983285 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.983391 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.983447 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 14:38:49.996062 21731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.991851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:37901 (local address 127.19.144.66:60069)
0808 14:38:49.992195 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:49.992208 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:49.992309 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:49.992584 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:49.992586 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:49.992592 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:38:49.993038 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.993045 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.994100 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.994103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.994746 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:49.994755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.995329 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.995332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:49.995409 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.995463 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:38:49.995770 (+ 307us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:38:49.995775 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 14:38:49.995825 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 14:38:49.995920 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":132,"threads_started":1}
I20260808 14:38:49.996047 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:49.991989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:60069 (local address 127.19.144.65:37901)
0808 14:38:49.992145 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:49.992150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:49.992288 (+ 138us) server_negotiation.cc:407] Connection header received
0808 14:38:49.992418 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:49.992421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:49.992477 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:49.992568 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:38:49.993158 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.993991 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.994864 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:49.995207 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:49.995309 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:49.995594 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:38:49.995686 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:38:49.995761 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:49.995837 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 14:38:49.995892 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 14:38:50.004170 21321 raft_consensus.cc:1261] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b [term 1 FOLLOWER]: Refusing update from remote peer 4d870ceb7bea47058f8be6b9ba840efe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:38:50.004626 21607 raft_consensus.cc:1261] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc [term 1 FOLLOWER]: Refusing update from remote peer 4d870ceb7bea47058f8be6b9ba840efe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:38:50.004740 21683 consensus_queue.cc:1035] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d1e42ebd8534d948044e34753d7553b" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:38:50.005013 21709 consensus_queue.cc:1035] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a88c49dd60b4e0d885c784c910ef3bc" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:38:50.031556 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:50.026412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:51493 (local address 127.19.144.66:35847)
0808 14:38:50.026551 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:50.026556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:50.026815 (+ 259us) server_negotiation.cc:407] Connection header received
0808 14:38:50.026918 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:50.026921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:50.026977 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:50.027061 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:38:50.027567 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:50.028341 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:50.029763 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:50.030140 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:50.030259 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:50.030644 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:38:50.030752 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:38:50.030830 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:50.031332 (+ 502us) server_negotiation.cc:299] Negotiation successful
0808 14:38:50.031399 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:38:50.032044 21735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:50.026285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:35847 (local address 127.19.144.65:51493)
0808 14:38:50.026728 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:50.026741 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:50.026836 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:50.027091 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:50.027094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:50.027103 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:38:50.027448 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:50.027456 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:50.028554 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:50.028557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:50.029649 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:50.029662 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:50.030310 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:50.030315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:50.030424 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:50.030490 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:38:50.031178 (+ 688us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:38:50.031188 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 14:38:50.031274 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 14:38:50.031379 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":136,"threads_started":1}
I20260808 14:38:50.060943 21162 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:38:50.061129 21162 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:38:50.063669 21162 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36441689f9b9466ca80f6d8c41576e0f: Sending DeleteTablet for 3 replicas of tablet acdb8d196a7943cb9012e4f8ea500060
I20260808 14:38:50.064455 21437 tablet_service.cc:1514] Processing DeleteTablet for tablet acdb8d196a7943cb9012e4f8ea500060 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:50 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:43780
I20260808 14:38:50.064541 21587 tablet_service.cc:1514] Processing DeleteTablet for tablet acdb8d196a7943cb9012e4f8ea500060 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:50 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:41360
I20260808 14:38:50.064680 21700 tablet_replica.cc:330] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe: stopping tablet replica
I20260808 14:38:50.064821 21700 raft_consensus.cc:2229] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:38:50.065140 21700 raft_consensus.cc:2258] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:38:50.065240 21301 tablet_service.cc:1514] Processing DeleteTablet for tablet acdb8d196a7943cb9012e4f8ea500060 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:50 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:50568
I20260808 14:38:50.065346 21702 tablet_replica.cc:330] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc: stopping tablet replica
I20260808 14:38:50.065434 21703 tablet_replica.cc:330] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b: stopping tablet replica
I20260808 14:38:50.065470 21702 raft_consensus.cc:2229] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:38:50.065562 21703 raft_consensus.cc:2229] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:38:50.065606 21702 raft_consensus.cc:2258] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:38:50.065785 21703 raft_consensus.cc:2258] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:38:50.066628 21702 ts_tablet_manager.cc:1905] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:50.066788 21703 ts_tablet_manager.cc:1905] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:50.069124 21702 ts_tablet_manager.cc:1918] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:38:50.069213 21702 log.cc:1196] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/acdb8d196a7943cb9012e4f8ea500060
I20260808 14:38:50.069523 21702 ts_tablet_manager.cc:1939] T acdb8d196a7943cb9012e4f8ea500060 P 1a88c49dd60b4e0d885c784c910ef3bc: Deleting consensus metadata
I20260808 14:38:50.069716 21703 ts_tablet_manager.cc:1918] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:38:50.069800 21703 log.cc:1196] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-0/wal/wals/acdb8d196a7943cb9012e4f8ea500060
I20260808 14:38:50.070063 21703 ts_tablet_manager.cc:1939] T acdb8d196a7943cb9012e4f8ea500060 P 7d1e42ebd8534d948044e34753d7553b: Deleting consensus metadata
I20260808 14:38:50.070744 21700 ts_tablet_manager.cc:1905] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:50.070717 21145 catalog_manager.cc:4841] TS 1a88c49dd60b4e0d885c784c910ef3bc (127.19.144.67:40601): tablet acdb8d196a7943cb9012e4f8ea500060 (table test-table [id=e6372e3dad6d44a48142fc1deb958f73]) successfully deleted
I20260808 14:38:50.070767 21144 catalog_manager.cc:4841] TS 7d1e42ebd8534d948044e34753d7553b (127.19.144.65:37901): tablet acdb8d196a7943cb9012e4f8ea500060 (table test-table [id=e6372e3dad6d44a48142fc1deb958f73]) successfully deleted
I20260808 14:38:50.072913 21700 ts_tablet_manager.cc:1918] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:38:50.073019 21700 log.cc:1196] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/acdb8d196a7943cb9012e4f8ea500060
I20260808 14:38:50.073271 21700 ts_tablet_manager.cc:1939] T acdb8d196a7943cb9012e4f8ea500060 P 4d870ceb7bea47058f8be6b9ba840efe: Deleting consensus metadata
I20260808 14:38:50.074050 21144 catalog_manager.cc:4841] TS 4d870ceb7bea47058f8be6b9ba840efe (127.19.144.66:35847): tablet acdb8d196a7943cb9012e4f8ea500060 (table test-table [id=e6372e3dad6d44a48142fc1deb958f73]) successfully deleted
Aug 08 14:38:50 dist-test-slave-p3t5 krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199929, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.144.65@KRBTEST.COM
I20260808 14:38:50.085574 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:50.074096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50586 (local address 127.19.144.65:37901)
0808 14:38:50.074225 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:50.074231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:50.074922 (+ 691us) server_negotiation.cc:407] Connection header received
0808 14:38:50.075090 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:50.075094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:50.075148 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:50.075223 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:50.076041 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:50.076668 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:50.077658 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:50.077908 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:50.078028 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:50.081684 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:50.081697 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:50.081713 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:50.081755 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:50.083591 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:50.084101 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:50.084107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:50.084108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:50.084199 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:50.084505 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:50.084508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:50.084510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:50.084781 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:50.085003 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:50.085240 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 14:38:50.085383 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 14:38:50.086287 21362 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50586
I20260808 14:38:50.096361 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:50.088302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44524 (local address 127.19.144.126:45381)
0808 14:38:50.088442 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:50.088455 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:50.088655 (+ 200us) server_negotiation.cc:407] Connection header received
0808 14:38:50.088813 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:50.088816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:50.088874 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:50.088963 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:50.089665 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:50.090256 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:50.091087 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:50.091333 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:50.091446 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:50.092887 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:50.092891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:50.092893 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:50.092929 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:50.094609 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:50.095060 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:50.095064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:50.095066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:50.095158 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:50.095454 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:50.095458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:50.095460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:50.095661 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:50.095809 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:50.096062 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 14:38:50.096187 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 14:38:50.096926 21162 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44524
I20260808 14:38:50.098459 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 21234
I20260808 14:38:50.110307 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 21377
I20260808 14:38:50.118000 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 21521
I20260808 14:38:50.127045 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 21128
2026-08-08T14:38:50Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2934 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:38:50 dist-test-slave-p3t5 krb5kdc[21746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:38:50 dist-test-slave-p3t5 krb5kdc[21746](info): set up 2 sockets
Aug 08 14:38:50 dist-test-slave-p3t5 krb5kdc[21746](info): commencing operation
krb5kdc: starting...
W20260808 14:38:51.229115 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:38:51 dist-test-slave-p3t5 krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:38:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:38:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:38:51.378274 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:45617
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:45617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:51.495344 21759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:51.495653 21759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:51.495697 21759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:51.499792 21759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:38:51.499864 21759 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 14:38:51.499881 21759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:51.499899 21759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:51.499917 21759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:38:51.499931 21759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:38:51.506044 21759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:45617
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:45617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:51.507182 21759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:51.508240 21759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:51.517732 21764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:51.517740 21765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:51.517917 21759 server_base.cc:1034] running on GCE node
W20260808 14:38:51.517732 21767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:51.518690 21759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:51.519927 21759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:51.521147 21759 hybrid_clock.cc:648] HybridClock initialized: now 1786199931521118 us; error 41 us; skew 500 ppm
Aug 08 14:38:51 dist-test-slave-p3t5 krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199931, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:51.524232 21759 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:38:51.525264 21759 webserver.cc:460] Webserver started at http://127.19.144.126:46345/ using document root <none> and password file <none>
I20260808 14:38:51.525944 21759 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:51.526044 21759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:51.526296 21759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:51.528465 21759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "6c14732906c34c74bf7e5044becee68c"
format_stamp: "Formatted at 2026-08-08 14:38:51 on dist-test-slave-p3t5"
server_key: "b81a795ee758f2b88b24cca8ab1b4d18"
server_key_iv: "7ab1c9b8148c1f0d26b31b2abbba0191"
server_key_version: "encryptionkey@0"
I20260808 14:38:51.529085 21759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "6c14732906c34c74bf7e5044becee68c"
format_stamp: "Formatted at 2026-08-08 14:38:51 on dist-test-slave-p3t5"
server_key: "b81a795ee758f2b88b24cca8ab1b4d18"
server_key_iv: "7ab1c9b8148c1f0d26b31b2abbba0191"
server_key_version: "encryptionkey@0"
I20260808 14:38:51.533218 21759 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:38:51.535976 21774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:51.537281 21759 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:38:51.537446 21759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "6c14732906c34c74bf7e5044becee68c"
format_stamp: "Formatted at 2026-08-08 14:38:51 on dist-test-slave-p3t5"
server_key: "b81a795ee758f2b88b24cca8ab1b4d18"
server_key_iv: "7ab1c9b8148c1f0d26b31b2abbba0191"
server_key_version: "encryptionkey@0"
I20260808 14:38:51.537575 21759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:51.557760 21759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:51.561038 21759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:51.561273 21759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:51.569851 21759 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:45617
I20260808 14:38:51.569868 21838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:45617 every 8 connection(s)
I20260808 14:38:51.570942 21759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:38:51.573974 21839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:51.576246 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 21759
I20260808 14:38:51.576346 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:38:51.576638 20033 external_mini_cluster.cc:1428] Setting key 92305374cd72d892a10ee68281316732
I20260808 14:38:51.580336 21839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c: Bootstrap starting.
I20260808 14:38:51.582813 21839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:51.583617 21839 log.cc:824] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:38:51 dist-test-slave-p3t5 krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:51.586304 21839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c: No bootstrap required, opened a new log
I20260808 14:38:51.591262 21839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c14732906c34c74bf7e5044becee68c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45617 } }
I20260808 14:38:51.591609 21839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:51.591684 21839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c14732906c34c74bf7e5044becee68c, State: Initialized, Role: FOLLOWER
I20260808 14:38:51.592218 21839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c14732906c34c74bf7e5044becee68c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45617 } }
I20260808 14:38:51.592381 21839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:38:51.592450 21839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:38:51.592549 21839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:51.592772 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:51.578216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48536 (local address 127.19.144.126:45617)
0808 14:38:51.578743 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:51.578761 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:51.578800 (+ 39us) server_negotiation.cc:407] Connection header received
0808 14:38:51.579483 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:51.579503 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:51.579866 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:51.580266 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:51.581179 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:51.582322 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:51.583316 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:51.583856 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:51.584033 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:51.587003 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:51.587013 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:51.587036 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:51.587083 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:51.589111 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:51.589656 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:51.589664 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:51.589671 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:51.589852 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:51.590205 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:51.590209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:51.590211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:51.590818 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:51.591044 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:51.591364 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 14:38:51.591710 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":169,"threads_started":1}
I20260808 14:38:51.593670 21839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c14732906c34c74bf7e5044becee68c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45617 } }
I20260808 14:38:51.594053 21839 leader_election.cc:302] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c14732906c34c74bf7e5044becee68c; no voters:
I20260808 14:38:51.594410 21839 leader_election.cc:288] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:38:51.594542 21844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:38:51.594780 21844 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [term 1 LEADER]: Becoming Leader. State: Replica: 6c14732906c34c74bf7e5044becee68c, State: Running, Role: LEADER
I20260808 14:38:51.595139 21844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c14732906c34c74bf7e5044becee68c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45617 } }
I20260808 14:38:51.595770 21839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:38:51.596769 21844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c14732906c34c74bf7e5044becee68c. Latest consensus state: current_term: 1 leader_uuid: "6c14732906c34c74bf7e5044becee68c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c14732906c34c74bf7e5044becee68c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45617 } } }
I20260808 14:38:51.596922 21844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [sys.catalog]: This master's current role is: LEADER
I20260808 14:38:51.597244 21846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c14732906c34c74bf7e5044becee68c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c14732906c34c74bf7e5044becee68c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45617 } } }
I20260808 14:38:51.597352 21846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c [sys.catalog]: This master's current role is: LEADER
I20260808 14:38:51.597412 21854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:38:51.600884 21854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:38:51.607153 21854 catalog_manager.cc:1347] Generated new cluster ID: 732c94936c354550b2a421f7a7831282
I20260808 14:38:51.607247 21854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:38:51.637167 21854 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:38:51.638347 21854 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:38:51.644651 21854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c: Generated new TSK 0
I20260808 14:38:51.645541 21854 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:38:51.678350 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:45617
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:51.800269 21866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:51.800601 21866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:51.800652 21866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:51.804929 21866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:51.805025 21866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:51.805070 21866 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 14:38:51.805085 21866 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:38:51.805164 21866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:38:51.810989 21866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.144.126:45617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:51.812400 21866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:51.813563 21866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:51.824477 21871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:51.824570 21866 server_base.cc:1034] running on GCE node
W20260808 14:38:51.824533 21874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:51.824486 21872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:51.825331 21866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:51.826117 21866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:51.827311 21866 hybrid_clock.cc:648] HybridClock initialized: now 1786199931827273 us; error 49 us; skew 500 ppm
Aug 08 14:38:51 dist-test-slave-p3t5 krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199931, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:51.830617 21866 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:38:51.831589 21866 webserver.cc:460] Webserver started at http://127.19.144.65:35071/ using document root <none> and password file <none>
I20260808 14:38:51.832182 21866 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:51.832247 21866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:51.832428 21866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:51.834686 21866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "0ee359e24adf43fa9f5f561785feda1d"
format_stamp: "Formatted at 2026-08-08 14:38:51 on dist-test-slave-p3t5"
server_key: "637d536f1e7b0672ce8380d81a51e481"
server_key_iv: "dade2371bff8adb01ef649c294597e90"
server_key_version: "encryptionkey@0"
I20260808 14:38:51.835240 21866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "0ee359e24adf43fa9f5f561785feda1d"
format_stamp: "Formatted at 2026-08-08 14:38:51 on dist-test-slave-p3t5"
server_key: "637d536f1e7b0672ce8380d81a51e481"
server_key_iv: "dade2371bff8adb01ef649c294597e90"
server_key_version: "encryptionkey@0"
I20260808 14:38:51.839588 21866 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:38:51.842625 21882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:51.844120 21866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:38:51.844292 21866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "0ee359e24adf43fa9f5f561785feda1d"
format_stamp: "Formatted at 2026-08-08 14:38:51 on dist-test-slave-p3t5"
server_key: "637d536f1e7b0672ce8380d81a51e481"
server_key_iv: "dade2371bff8adb01ef649c294597e90"
server_key_version: "encryptionkey@0"
I20260808 14:38:51.844436 21866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:51.872058 21866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:51.875777 21866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:51.876040 21866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:51.877740 21866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:38:51.877827 21866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:51.877904 21866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:38:51.877956 21866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 14:38:51 dist-test-slave-p3t5 krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199931, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:51.889958 21866 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:32785
I20260808 14:38:51.889979 22005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:32785 every 8 connection(s)
I20260808 14:38:51.891362 21866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:38:51.894522 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:51.878811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:37437 (local address 127.19.144.126:45617)
0808 14:38:51.878985 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:51.878991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:51.880074 (+ 1083us) server_negotiation.cc:407] Connection header received
0808 14:38:51.881520 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:51.881527 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:51.881603 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:51.881768 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:51.883718 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:51.884366 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:51.885915 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:51.886175 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:51.886302 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:51.889770 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:51.889776 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:51.889779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:51.889823 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:51.891763 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:51.892490 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:51.892494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:51.892496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:51.892602 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:51.893048 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:51.893051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:51.893052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:51.893270 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:51.893373 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:51.894231 (+ 858us) server_negotiation.cc:299] Negotiation successful
0808 14:38:51.894387 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 14:38:51.895460 21891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:51.879289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45617 (local address 127.19.144.65:37437)
0808 14:38:51.879862 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:51.879930 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:51.881182 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:51.882003 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:51.882013 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:51.882614 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:51.883481 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:51.883508 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:51.884520 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:51.884524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:51.885768 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:51.885781 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:51.886366 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:51.886371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:51.886598 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:51.887375 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:51.887394 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:51.889457 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:51.891938 (+ 2481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:51.891946 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:51.891965 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:51.892357 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:51.892740 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:51.892752 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:51.892754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:51.892933 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:51.893404 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:51.893410 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:51.893914 (+ 504us) client_negotiation.cc:769] Sending connection context
0808 14:38:51.894112 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 14:38:51.894423 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":119,"threads_started":1}
I20260808 14:38:51.897269 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 21866
I20260808 14:38:51.897435 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:38:51.897573 22006 heartbeater.cc:347] Connected to a master server at 127.19.144.126:45617
I20260808 14:38:51.897774 22006 heartbeater.cc:464] Registering TS with master...
I20260808 14:38:51.897851 20033 external_mini_cluster.cc:1428] Setting key 4957794534512c58e4a9aaf2307bceab
I20260808 14:38:51.898487 22006 heartbeater.cc:511] Master 127.19.144.126:45617 requested a full tablet report, sending...
I20260808 14:38:51.901046 21790 ts_manager.cc:194] Registered new tserver with Master: 0ee359e24adf43fa9f5f561785feda1d (127.19.144.65:32785)
I20260808 14:38:51.902766 21790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:37437
I20260808 14:38:51.908455 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:51.900469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:41267 (local address 127.19.144.126:45617)
0808 14:38:51.900611 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:51.900618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:51.900635 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:38:51.900680 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:51.900686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:51.900744 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:51.900837 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:51.901420 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:51.902242 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:51.903192 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:51.903435 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:51.903576 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:51.904974 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:51.904978 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:51.904981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:51.905014 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:51.906685 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:51.907168 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:51.907172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:51.907174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:51.907267 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:51.907571 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:51.907575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:51.907577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:51.907776 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:51.907886 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:51.908155 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 14:38:51.908273 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 14:38:51.908473 22013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:51.900053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45617 (local address 127.19.144.65:41267)
0808 14:38:51.900413 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:51.900430 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:51.900529 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:51.900878 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:51.900881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:51.901139 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:51.901284 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:51.901293 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:51.902392 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:51.902395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:51.903067 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:51.903077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:51.903643 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:51.903646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:51.903751 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:51.904410 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:51.904432 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:51.904809 (+ 377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:51.906828 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:51.906832 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:51.906834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:51.907058 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:51.907371 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:51.907375 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:51.907377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:51.907474 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:51.907875 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:51.907879 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:51.907983 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 14:38:51.908094 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 14:38:51.908262 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":143,"threads_started":1}
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:38:51.932986 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:45617
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:52.055007 22016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:52.055333 22016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:52.055389 22016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:52.059571 22016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:52.059659 22016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:52.059703 22016 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 14:38:52.059718 22016 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:38:52.059752 22016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:38:52.065570 22016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.144.126:45617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:52.066850 22016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:52.067932 22016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:52.078444 22024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:52.078444 22021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:52.078450 22022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:52.078737 22016 server_base.cc:1034] running on GCE node
I20260808 14:38:52.079349 22016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:52.080065 22016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:52.081306 22016 hybrid_clock.cc:648] HybridClock initialized: now 1786199932081285 us; error 33 us; skew 500 ppm
Aug 08 14:38:52 dist-test-slave-p3t5 krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199932, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:52.084640 22016 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:38:52.085711 22016 webserver.cc:460] Webserver started at http://127.19.144.66:39333/ using document root <none> and password file <none>
I20260808 14:38:52.086383 22016 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:52.086493 22016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:52.086762 22016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:52.089011 22016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca"
format_stamp: "Formatted at 2026-08-08 14:38:52 on dist-test-slave-p3t5"
server_key: "b3fb740ef308e5c1b73429632218bcb4"
server_key_iv: "93398c918cd540fd542bd91bec99f75a"
server_key_version: "encryptionkey@0"
I20260808 14:38:52.089689 22016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca"
format_stamp: "Formatted at 2026-08-08 14:38:52 on dist-test-slave-p3t5"
server_key: "b3fb740ef308e5c1b73429632218bcb4"
server_key_iv: "93398c918cd540fd542bd91bec99f75a"
server_key_version: "encryptionkey@0"
I20260808 14:38:52.094018 22016 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:38:52.096742 22031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:52.098366 22016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:38:52.098516 22016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca"
format_stamp: "Formatted at 2026-08-08 14:38:52 on dist-test-slave-p3t5"
server_key: "b3fb740ef308e5c1b73429632218bcb4"
server_key_iv: "93398c918cd540fd542bd91bec99f75a"
server_key_version: "encryptionkey@0"
I20260808 14:38:52.098647 22016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:52.127655 22016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:52.131548 22016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:52.131783 22016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:52.133491 22016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:38:52.133575 22016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:52.133677 22016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:38:52.133731 22016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 14:38:52 dist-test-slave-p3t5 krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199932, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:52.145280 22016 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:33193
I20260808 14:38:52.145354 22149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:33193 every 8 connection(s)
I20260808 14:38:52.146653 22016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:38:52.150315 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.134451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:43623 (local address 127.19.144.126:45617)
0808 14:38:52.134643 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.134649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.136122 (+ 1473us) server_negotiation.cc:407] Connection header received
0808 14:38:52.137726 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.137730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.137806 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.137919 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:52.139448 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.140003 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.141569 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.141897 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.142048 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.145422 (+ 3374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:52.145427 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:52.145430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:52.145470 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:52.148081 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:52.148677 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:52.148679 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:52.148680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:52.148771 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:52.149144 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:52.149146 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:52.149147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:52.149400 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:52.149475 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.150067 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.150185 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 14:38:52.151288 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.135328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45617 (local address 127.19.144.66:43623)
0808 14:38:52.135930 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:52.135994 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:52.137398 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:52.138170 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:52.138178 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:52.138593 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:52.139265 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.139285 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.140169 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.140173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.141406 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.141420 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.142078 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.142084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.142298 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.143124 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:52.143143 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:52.145136 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:52.148255 (+ 3119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:52.148268 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:52.148281 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:52.148573 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:52.148908 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:52.148910 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:52.148911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:52.149060 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:52.149491 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:52.149497 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:52.149830 (+ 333us) client_negotiation.cc:769] Sending connection context
0808 14:38:52.150011 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 14:38:52.150300 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":165,"threads_started":1}
I20260808 14:38:52.152004 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 22016
I20260808 14:38:52.152127 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:38:52.152463 20033 external_mini_cluster.cc:1428] Setting key 99d15e24d922cfeb9d1e03490832969e
I20260808 14:38:52.152809 22150 heartbeater.cc:347] Connected to a master server at 127.19.144.126:45617
I20260808 14:38:52.153013 22150 heartbeater.cc:464] Registering TS with master...
I20260808 14:38:52.153723 22150 heartbeater.cc:511] Master 127.19.144.126:45617 requested a full tablet report, sending...
I20260808 14:38:52.154953 21790 ts_manager.cc:194] Registered new tserver with Master: 6d5bb3c8a0e741f98aad497ffa3e83ca (127.19.144.66:33193)
I20260808 14:38:52.155694 21790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:43623
I20260808 14:38:52.164095 22156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.155273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45617 (local address 127.19.144.66:40971)
0808 14:38:52.156122 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:52.156135 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:52.156275 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:52.156576 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:52.156579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:52.156822 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:52.156960 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.156968 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.158103 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.158106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.158667 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.158675 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.159338 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.159341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.159431 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.159994 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:52.160012 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:52.160368 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:52.162458 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:52.162461 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:52.162462 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:52.162675 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:52.162998 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:52.163000 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:52.163002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:52.163079 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:52.163635 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:52.163637 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:52.163720 (+ 83us) client_negotiation.cc:769] Sending connection context
0808 14:38:52.163801 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 14:38:52.163933 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":118,"threads_started":1}
I20260808 14:38:52.164256 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.155367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:40971 (local address 127.19.144.126:45617)
0808 14:38:52.155978 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.155981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.156223 (+ 242us) server_negotiation.cc:407] Connection header received
0808 14:38:52.156376 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.156382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.156447 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.156538 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:52.157101 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.157957 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.158796 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.159197 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.159343 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.160529 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:52.160534 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:52.160540 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:52.160573 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:52.162327 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:52.162782 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:52.162786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:52.162789 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:52.162897 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:52.163174 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:52.163179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:52.163183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:52.163534 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:52.163654 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.163905 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.164051 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545}
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:38:52.189373 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:45617
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:52.314728 22159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:52.315028 22159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:52.315080 22159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:52.319193 22159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:52.319263 22159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:52.319305 22159 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 14:38:52.319319 22159 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:38:52.319353 22159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:38:52.324698 22159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.144.126:45617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:52.325892 22159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:52.327033 22159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:52.337301 22164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:52.337356 22167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:52.337443 22159 server_base.cc:1034] running on GCE node
W20260808 14:38:52.337301 22165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:52.338110 22159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:52.338833 22159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:52.340052 22159 hybrid_clock.cc:648] HybridClock initialized: now 1786199932340015 us; error 48 us; skew 500 ppm
Aug 08 14:38:52 dist-test-slave-p3t5 krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199932, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:52.343384 22159 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:38:52.344345 22159 webserver.cc:460] Webserver started at http://127.19.144.67:39869/ using document root <none> and password file <none>
I20260808 14:38:52.344954 22159 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:52.345052 22159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:52.345299 22159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:52.347548 22159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "9a42068d36d24e25a3296e6a4eb8f918"
format_stamp: "Formatted at 2026-08-08 14:38:52 on dist-test-slave-p3t5"
server_key: "6a07a0fe2bd90fd46c49803cd2010438"
server_key_iv: "18f36e7d3d180808b081cd6e5b58eb1d"
server_key_version: "encryptionkey@0"
I20260808 14:38:52.348263 22159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "9a42068d36d24e25a3296e6a4eb8f918"
format_stamp: "Formatted at 2026-08-08 14:38:52 on dist-test-slave-p3t5"
server_key: "6a07a0fe2bd90fd46c49803cd2010438"
server_key_iv: "18f36e7d3d180808b081cd6e5b58eb1d"
server_key_version: "encryptionkey@0"
I20260808 14:38:52.352480 22159 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:38:52.355319 22174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:52.356716 22159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 14:38:52.356885 22159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "9a42068d36d24e25a3296e6a4eb8f918"
format_stamp: "Formatted at 2026-08-08 14:38:52 on dist-test-slave-p3t5"
server_key: "6a07a0fe2bd90fd46c49803cd2010438"
server_key_iv: "18f36e7d3d180808b081cd6e5b58eb1d"
server_key_version: "encryptionkey@0"
I20260808 14:38:52.357024 22159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:52.378567 22159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:52.382571 22159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:52.382840 22159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:52.384543 22159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:38:52.384630 22159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:52.384718 22159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:38:52.384769 22159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 14:38:52 dist-test-slave-p3t5 krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199932, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:52.397746 22159 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:34289
I20260808 14:38:52.397766 22299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:34289 every 8 connection(s)
I20260808 14:38:52.399180 22159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:38:52.401569 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.385558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:52711 (local address 127.19.144.126:45617)
0808 14:38:52.385716 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.385720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.387000 (+ 1280us) server_negotiation.cc:407] Connection header received
0808 14:38:52.388326 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.388332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.388418 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.388547 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:52.390171 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.390732 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.392112 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.392475 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.392594 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.396889 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:52.396892 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:52.396894 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:52.396925 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:52.398974 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:52.399769 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:52.399773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:52.399774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:52.399894 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:52.400333 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:52.400336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:52.400338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:52.400566 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:52.400688 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.401231 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.401387 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 14:38:52.402575 22183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.386223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45617 (local address 127.19.144.67:52711)
0808 14:38:52.386819 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:52.386873 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:52.387990 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:52.388769 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:52.388781 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:52.389319 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:52.389999 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.390014 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.390881 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.390884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.391973 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.391984 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.392657 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.392661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.392896 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.393787 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:52.393806 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:52.396596 (+ 2790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:52.399181 (+ 2585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:52.399186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:52.399200 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:52.399609 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:52.400012 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:52.400015 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:52.400017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:52.400212 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:52.400686 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:52.400693 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:52.400968 (+ 275us) client_negotiation.cc:769] Sending connection context
0808 14:38:52.401166 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 14:38:52.401527 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":146,"threads_started":1}
I20260808 14:38:52.404212 22300 heartbeater.cc:347] Connected to a master server at 127.19.144.126:45617
I20260808 14:38:52.404415 22300 heartbeater.cc:464] Registering TS with master...
I20260808 14:38:52.405045 22300 heartbeater.cc:511] Master 127.19.144.126:45617 requested a full tablet report, sending...
I20260808 14:38:52.406380 21790 ts_manager.cc:194] Registered new tserver with Master: 9a42068d36d24e25a3296e6a4eb8f918 (127.19.144.67:34289)
I20260808 14:38:52.407171 21790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:52711
I20260808 14:38:52.408354 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 22159
I20260808 14:38:52.408495 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:38:52.409107 20033 external_mini_cluster.cc:1428] Setting key 402d8ad401f325fe4663aa16f82b2e12
I20260808 14:38:52.411628 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:38:52.415969 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.406924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:37615 (local address 127.19.144.126:45617)
0808 14:38:52.407061 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.407066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.407325 (+ 259us) server_negotiation.cc:407] Connection header received
0808 14:38:52.407501 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.407504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.407551 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.407640 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:52.408162 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.409222 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.410315 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.410607 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.410708 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.412196 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:52.412200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:52.412203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:52.412237 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:52.414008 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:52.414559 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:52.414564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:52.414567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:52.414680 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:52.414967 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:52.414972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:52.414975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:52.415235 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:52.415361 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.415677 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.415799 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:38:52.416059 22306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.406795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45617 (local address 127.19.144.67:37615)
0808 14:38:52.407238 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:52.407253 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:52.407409 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:52.407659 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:52.407662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:52.407880 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:52.407989 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.407995 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.409402 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.409406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.410184 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.410195 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.410795 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.410799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.410944 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.411604 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:52.411616 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:52.412051 (+ 435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:52.414152 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:52.414155 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:52.414159 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:52.414444 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:52.414785 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:52.414787 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:52.414788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:52.414873 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:52.415382 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:52.415385 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:52.415480 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 14:38:52.415581 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 14:38:52.415767 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":127,"threads_started":1}
Aug 08 14:38:52 dist-test-slave-p3t5 krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 14:38:52 dist-test-slave-p3t5 krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:52.443326 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.434607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48552 (local address 127.19.144.126:45617)
0808 14:38:52.434779 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.434783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.434797 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:38:52.434843 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.434846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.434886 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.434987 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:52.435750 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.436298 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.437175 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.437403 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.437516 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.440132 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:52.440136 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:52.440138 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:52.440165 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:52.441552 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:52.442041 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:52.442045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:52.442046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:52.442134 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:52.442438 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:52.442442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:52.442444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:52.442638 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:52.442793 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.443023 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.443145 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:38:52.446457 21790 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:38:52.449970 21790 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:38:52.465997 22318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.459618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:34289 (local address 127.0.0.1:56840)
0808 14:38:52.460506 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:52.460565 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:52.460708 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:52.461361 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:52.461367 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:52.461398 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:52.461548 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.461556 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.462859 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.462864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.464234 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.464246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.465342 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.465346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.465464 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.465524 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 14:38:52.465663 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 14:38:52.465822 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":38,"thread_start_us":99,"threads_started":1}
I20260808 14:38:52.466198 22320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.459738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56840 (local address 127.19.144.67:34289)
0808 14:38:52.460471 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.460483 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.460685 (+ 202us) server_negotiation.cc:407] Connection header received
0808 14:38:52.460940 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.460947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.461179 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.461343 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:52.461688 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.462744 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.464386 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.465201 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.465325 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.465505 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.465833 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.466027 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":115,"threads_started":1}
I20260808 14:38:52.466416 22319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.459991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:32785 (local address 127.0.0.1:52656)
0808 14:38:52.461346 (+ 1355us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:52.461359 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:52.461442 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:52.462235 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:52.462238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:52.462255 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:52.462346 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.462352 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.463658 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.463661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.464653 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.464664 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.466010 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.466016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.466129 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.466152 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 14:38:52.466225 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 14:38:52.466286 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1297,"mutex_wait_us":27,"thread_start_us":106,"threads_started":1}
I20260808 14:38:52.466753 22321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.459689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:33193 (local address 127.0.0.1:34262)
0808 14:38:52.460977 (+ 1288us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:52.460991 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:52.461073 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:52.462009 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:52.462013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:52.462030 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:52.462130 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.462136 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.463513 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.463516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.464656 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.464664 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.465597 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.465604 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.465732 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.465757 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 14:38:52.465827 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 14:38:52.465907 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1223,"mutex_wait_us":59,"thread_start_us":122,"threads_started":1}
I20260808 14:38:52.466966 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.460927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52656 (local address 127.19.144.65:32785)
0808 14:38:52.461329 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.461336 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.461537 (+ 201us) server_negotiation.cc:407] Connection header received
0808 14:38:52.461665 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.461672 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.461876 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.462107 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:52.462469 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.463543 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.464792 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.465567 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.465732 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.466626 (+ 894us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.466686 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.466841 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":89,"threads_started":1}
I20260808 14:38:52.467504 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.460501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34262 (local address 127.19.144.66:33193)
0808 14:38:52.461523 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.461534 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.461556 (+ 22us) server_negotiation.cc:407] Connection header received
0808 14:38:52.461663 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.461671 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.461834 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.461991 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:52.462245 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.463380 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.464774 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.465425 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.465529 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.465737 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.466119 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.467365 (+ 1246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":926,"thread_start_us":142,"threads_started":1}
I20260808 14:38:52.469456 21933 tablet_service.cc:1467] Processing CreateTablet for tablet 040c68e49ec34a3ea19e7b76bd550977 (DEFAULT_TABLE table=test-table [id=ca9a22b1d9324996a4398aa16a8097a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:52.469460 22227 tablet_service.cc:1467] Processing CreateTablet for tablet 040c68e49ec34a3ea19e7b76bd550977 (DEFAULT_TABLE table=test-table [id=ca9a22b1d9324996a4398aa16a8097a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:52.470201 22081 tablet_service.cc:1467] Processing CreateTablet for tablet 040c68e49ec34a3ea19e7b76bd550977 (DEFAULT_TABLE table=test-table [id=ca9a22b1d9324996a4398aa16a8097a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:52.470626 21933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 040c68e49ec34a3ea19e7b76bd550977. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:52.470638 22227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 040c68e49ec34a3ea19e7b76bd550977. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:52.471207 22081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 040c68e49ec34a3ea19e7b76bd550977. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:52.477185 22324 tablet_bootstrap.cc:492] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d: Bootstrap starting.
I20260808 14:38:52.479391 22325 tablet_bootstrap.cc:492] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918: Bootstrap starting.
I20260808 14:38:52.479645 22324 tablet_bootstrap.cc:654] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:52.480402 22324 log.cc:824] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d: Log is configured to *not* fsync() on all Append() calls
I20260808 14:38:52.481055 22326 tablet_bootstrap.cc:492] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Bootstrap starting.
I20260808 14:38:52.481542 22325 tablet_bootstrap.cc:654] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:52.482467 22325 log.cc:824] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918: Log is configured to *not* fsync() on all Append() calls
I20260808 14:38:52.482746 22324 tablet_bootstrap.cc:492] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d: No bootstrap required, opened a new log
I20260808 14:38:52.483004 22324 ts_tablet_manager.cc:1397] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:38:52.483533 22326 tablet_bootstrap.cc:654] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:52.484373 22326 log.cc:824] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Log is configured to *not* fsync() on all Append() calls
I20260808 14:38:52.484743 22325 tablet_bootstrap.cc:492] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918: No bootstrap required, opened a new log
I20260808 14:38:52.484972 22325 ts_tablet_manager.cc:1397] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:38:52.486588 22326 tablet_bootstrap.cc:492] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca: No bootstrap required, opened a new log
I20260808 14:38:52.486799 22326 ts_tablet_manager.cc:1397] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:38:52.487304 22324 raft_consensus.cc:348] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } }
I20260808 14:38:52.487561 22324 raft_consensus.cc:374] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:52.487615 22324 raft_consensus.cc:729] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee359e24adf43fa9f5f561785feda1d, State: Initialized, Role: FOLLOWER
I20260808 14:38:52.488078 22324 consensus_queue.cc:260] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } }
I20260808 14:38:52.488741 22006 heartbeater.cc:503] Master 127.19.144.126:45617 was elected leader, sending a full tablet report...
I20260808 14:38:52.489118 22324 ts_tablet_manager.cc:1428] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 14:38:52.490068 22325 raft_consensus.cc:348] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } }
I20260808 14:38:52.490338 22325 raft_consensus.cc:374] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:52.490399 22325 raft_consensus.cc:729] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a42068d36d24e25a3296e6a4eb8f918, State: Initialized, Role: FOLLOWER
I20260808 14:38:52.490659 22326 raft_consensus.cc:348] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } }
I20260808 14:38:52.490906 22326 raft_consensus.cc:374] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:52.490967 22326 raft_consensus.cc:729] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5bb3c8a0e741f98aad497ffa3e83ca, State: Initialized, Role: FOLLOWER
I20260808 14:38:52.490885 22325 consensus_queue.cc:260] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } }
I20260808 14:38:52.491400 22326 consensus_queue.cc:260] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } }
I20260808 14:38:52.491627 22300 heartbeater.cc:503] Master 127.19.144.126:45617 was elected leader, sending a full tablet report...
I20260808 14:38:52.491932 22325 ts_tablet_manager.cc:1428] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 14:38:52.492311 22326 ts_tablet_manager.cc:1428] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 14:38:52.492631 22150 heartbeater.cc:503] Master 127.19.144.126:45617 was elected leader, sending a full tablet report...
I20260808 14:38:52.502717 22332 raft_consensus.cc:482] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:38:52.502957 22332 raft_consensus.cc:504] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } }
I20260808 14:38:52.504315 22332 leader_election.cc:288] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a42068d36d24e25a3296e6a4eb8f918 (127.19.144.67:34289), 0ee359e24adf43fa9f5f561785feda1d (127.19.144.65:32785)
I20260808 14:38:52.508006 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.504440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:34289 (local address 127.19.144.66:42409)
0808 14:38:52.504639 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:52.504658 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:52.504755 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:52.505049 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:52.505052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:52.505071 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:52.505193 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.505198 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.506064 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.506069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.506877 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.506886 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.507553 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.507556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.507674 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.507700 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 14:38:52.507780 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 14:38:52.507875 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260808 14:38:52.508080 22320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.504567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:42409 (local address 127.19.144.67:34289)
0808 14:38:52.504717 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.504721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.504737 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:38:52.504899 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.504901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.504943 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.505034 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:52.505336 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.505946 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.507017 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.507426 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.507562 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.507620 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.507836 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.507937 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 14:38:52.508724 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "040c68e49ec34a3ea19e7b76bd550977" candidate_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a42068d36d24e25a3296e6a4eb8f918" is_pre_election: true
I20260808 14:38:52.509091 22252 raft_consensus.cc:2454] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5bb3c8a0e741f98aad497ffa3e83ca in term 0.
I20260808 14:38:52.509214 22333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.504641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:32785 (local address 127.19.144.66:42777)
0808 14:38:52.505060 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:52.505072 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:52.505202 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:52.505552 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:52.505555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:52.505572 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:52.506097 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.506105 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.506883 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.506888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.507831 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.507843 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.508793 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.508797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.508896 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.508920 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:38:52.509002 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 14:38:52.509065 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":222,"threads_started":1}
I20260808 14:38:52.509227 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.504712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:42777 (local address 127.19.144.65:32785)
0808 14:38:52.504851 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.504858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.505167 (+ 309us) server_negotiation.cc:407] Connection header received
0808 14:38:52.505394 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.505403 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.505448 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.505538 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:52.506228 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.506754 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.507981 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.508635 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.508764 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.508818 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.509032 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.509106 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:38:52.509685 22033 leader_election.cc:302] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5bb3c8a0e741f98aad497ffa3e83ca, 9a42068d36d24e25a3296e6a4eb8f918; no voters:
I20260808 14:38:52.509874 21959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "040c68e49ec34a3ea19e7b76bd550977" candidate_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee359e24adf43fa9f5f561785feda1d" is_pre_election: true
I20260808 14:38:52.509984 22332 raft_consensus.cc:2781] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:38:52.510113 22332 raft_consensus.cc:482] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:38:52.510169 22332 raft_consensus.cc:3037] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:52.510207 21959 raft_consensus.cc:2454] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5bb3c8a0e741f98aad497ffa3e83ca in term 0.
I20260808 14:38:52.511366 22332 raft_consensus.cc:504] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } }
I20260808 14:38:52.511762 22332 leader_election.cc:288] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [CANDIDATE]: Term 1 election: Requested vote from peers 9a42068d36d24e25a3296e6a4eb8f918 (127.19.144.67:34289), 0ee359e24adf43fa9f5f561785feda1d (127.19.144.65:32785)
I20260808 14:38:52.512089 21959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "040c68e49ec34a3ea19e7b76bd550977" candidate_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee359e24adf43fa9f5f561785feda1d"
I20260808 14:38:52.512089 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "040c68e49ec34a3ea19e7b76bd550977" candidate_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a42068d36d24e25a3296e6a4eb8f918"
I20260808 14:38:52.512220 22252 raft_consensus.cc:3037] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:52.512220 21959 raft_consensus.cc:3037] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:52.513602 21959 raft_consensus.cc:2454] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5bb3c8a0e741f98aad497ffa3e83ca in term 1.
I20260808 14:38:52.513602 22252 raft_consensus.cc:2454] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5bb3c8a0e741f98aad497ffa3e83ca in term 1.
I20260808 14:38:52.513974 22032 leader_election.cc:302] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee359e24adf43fa9f5f561785feda1d, 6d5bb3c8a0e741f98aad497ffa3e83ca; no voters:
I20260808 14:38:52.514154 22332 raft_consensus.cc:2781] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:38:52.514402 22332 raft_consensus.cc:686] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 LEADER]: Becoming Leader. State: Replica: 6d5bb3c8a0e741f98aad497ffa3e83ca, State: Running, Role: LEADER
I20260808 14:38:52.514744 22332 consensus_queue.cc:237] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } }
I20260808 14:38:52.518458 21790 catalog_manager.cc:5495] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d5bb3c8a0e741f98aad497ffa3e83ca (127.19.144.66). New cstate: current_term: 1 leader_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } health_report { overall_health: UNKNOWN } } }
Aug 08 14:38:52 dist-test-slave-p3t5 krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199931, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:52.578670 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.560472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48558 (local address 127.19.144.126:45617)
0808 14:38:52.560648 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.560653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.561158 (+ 505us) server_negotiation.cc:407] Connection header received
0808 14:38:52.561377 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.561380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.561451 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.561569 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:52.562676 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.563398 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.564639 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.564998 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.565139 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.574211 (+ 9072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:52.574215 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:52.574218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:52.574247 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:52.576438 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:52.577003 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:52.577008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:52.577010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:52.577110 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:52.577430 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:52.577436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:52.577438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:52.577740 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:52.577867 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.578209 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.578496 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":146,"server-negotiator.queue_time_us":65}
I20260808 14:38:52.578670 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.560414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45617 (local address 127.0.0.1:48558)
0808 14:38:52.561046 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:52.561069 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:52.561245 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:52.561678 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:52.561685 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:52.562369 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:52.562532 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.562540 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.563542 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.563547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.564455 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.564470 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.566459 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.566463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.566572 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.571629 (+ 5057us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:52.571649 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:52.574005 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:52.576571 (+ 2566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:52.576584 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:52.576591 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:52.576879 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:52.577218 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:52.577221 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:52.577223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:52.577329 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:52.577866 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:52.577876 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:52.578013 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 14:38:52.578151 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 14:38:52.578435 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":27,"thread_start_us":165,"threads_started":1}
I20260808 14:38:52.583599 21790 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:48558:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 14:38:52.584440 21790 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:38:52.590291 21933 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2dcadc10634d0e8b47d2159dec4335 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a83affcdd4bd418d8303b666eacf073c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:38:52.590303 22081 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2dcadc10634d0e8b47d2159dec4335 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a83affcdd4bd418d8303b666eacf073c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:38:52.590557 22227 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2dcadc10634d0e8b47d2159dec4335 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a83affcdd4bd418d8303b666eacf073c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:38:52.591102 21933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2dcadc10634d0e8b47d2159dec4335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:52.591149 22081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2dcadc10634d0e8b47d2159dec4335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:52.591192 22227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2dcadc10634d0e8b47d2159dec4335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:52.594089 22324 tablet_bootstrap.cc:492] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d: Bootstrap starting.
I20260808 14:38:52.594537 22326 tablet_bootstrap.cc:492] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Bootstrap starting.
I20260808 14:38:52.595355 22324 tablet_bootstrap.cc:654] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:52.595561 22326 tablet_bootstrap.cc:654] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:52.596252 22325 tablet_bootstrap.cc:492] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918: Bootstrap starting.
I20260808 14:38:52.597072 22324 tablet_bootstrap.cc:492] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d: No bootstrap required, opened a new log
I20260808 14:38:52.597211 22324 ts_tablet_manager.cc:1397] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260808 14:38:52.597371 22325 tablet_bootstrap.cc:654] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:52.597834 22324 raft_consensus.cc:348] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:52.597993 22324 raft_consensus.cc:374] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:52.598040 22324 raft_consensus.cc:729] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee359e24adf43fa9f5f561785feda1d, State: Initialized, Role: FOLLOWER
I20260808 14:38:52.598186 22324 consensus_queue.cc:260] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:52.598526 22324 ts_tablet_manager.cc:1428] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 14:38:52.598544 22330 tablet_replica.cc:439] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } }
I20260808 14:38:52.598747 22330 tablet_replica.cc:442] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:38:52.598973 22326 tablet_bootstrap.cc:492] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca: No bootstrap required, opened a new log
I20260808 14:38:52.599048 22326 ts_tablet_manager.cc:1397] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 14:38:52.599234 22325 tablet_bootstrap.cc:492] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918: No bootstrap required, opened a new log
I20260808 14:38:52.599321 22325 ts_tablet_manager.cc:1397] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:38:52.599565 22326 raft_consensus.cc:348] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:52.599706 22326 raft_consensus.cc:374] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:52.599781 22326 raft_consensus.cc:729] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5bb3c8a0e741f98aad497ffa3e83ca, State: Initialized, Role: FOLLOWER
I20260808 14:38:52.599829 22325 raft_consensus.cc:348] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:52.599949 22325 raft_consensus.cc:374] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:52.599990 22325 raft_consensus.cc:729] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a42068d36d24e25a3296e6a4eb8f918, State: Initialized, Role: FOLLOWER
I20260808 14:38:52.599938 22326 consensus_queue.cc:260] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:52.600124 22325 consensus_queue.cc:260] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:52.600236 22332 tablet_replica.cc:439] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } }
I20260808 14:38:52.600391 22332 tablet_replica.cc:442] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:38:52.600428 22325 ts_tablet_manager.cc:1428] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 14:38:52.600461 22326 ts_tablet_manager.cc:1428] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 14:38:52.600704 22331 tablet_replica.cc:439] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } }
I20260808 14:38:52.600826 22331 tablet_replica.cc:442] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 14:38:52.643105 22007 tablet.cc:2369] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:38:52.648144 22151 tablet.cc:2369] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:38:52.650327 22301 tablet.cc:2369] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:38:52.765550 22331 raft_consensus.cc:482] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:38:52.765793 22331 raft_consensus.cc:504] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:52.767180 22331 leader_election.cc:288] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee359e24adf43fa9f5f561785feda1d (127.19.144.65:32785), 6d5bb3c8a0e741f98aad497ffa3e83ca (127.19.144.66:33193)
I20260808 14:38:52.772186 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.767649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:40237 (local address 127.19.144.66:33193)
0808 14:38:52.767805 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.767810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.767832 (+ 22us) server_negotiation.cc:407] Connection header received
0808 14:38:52.767902 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.767907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.767961 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.768061 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:52.768397 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.769184 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.770751 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.771489 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.771659 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.771721 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.771949 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.772034 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 14:38:52.772186 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.767671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:36395 (local address 127.19.144.65:32785)
0808 14:38:52.767808 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.767813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.768234 (+ 421us) server_negotiation.cc:407] Connection header received
0808 14:38:52.768314 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.768317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.768386 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.768467 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:52.768731 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.769439 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.770883 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.771560 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.771681 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.771724 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.771949 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.772034 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 14:38:52.772199 22183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.767506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:33193 (local address 127.19.144.67:40237)
0808 14:38:52.767656 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:52.767675 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:52.767792 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:52.768092 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:52.768095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:52.768116 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:52.768270 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.768279 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.769299 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.769302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.770585 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.770598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.771656 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.771660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.771779 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.771815 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 14:38:52.771930 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 14:38:52.772028 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":28}
I20260808 14:38:52.772207 22345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.767509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:32785 (local address 127.19.144.67:36395)
0808 14:38:52.768123 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:52.768136 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:52.768232 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:52.768493 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:52.768495 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:52.768512 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:52.768615 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.768619 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.769548 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.769552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.770730 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.770738 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.771695 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.771699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.771783 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.771811 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 14:38:52.771896 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 14:38:52.772028 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":256,"thread_start_us":129,"threads_started":1}
I20260808 14:38:52.772913 21959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2dcadc10634d0e8b47d2159dec4335" candidate_uuid: "9a42068d36d24e25a3296e6a4eb8f918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee359e24adf43fa9f5f561785feda1d" is_pre_election: true
I20260808 14:38:52.773000 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2dcadc10634d0e8b47d2159dec4335" candidate_uuid: "9a42068d36d24e25a3296e6a4eb8f918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" is_pre_election: true
I20260808 14:38:52.773104 21959 raft_consensus.cc:2454] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a42068d36d24e25a3296e6a4eb8f918 in term 0.
I20260808 14:38:52.773221 22102 raft_consensus.cc:2454] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a42068d36d24e25a3296e6a4eb8f918 in term 0.
I20260808 14:38:52.773557 22175 leader_election.cc:302] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee359e24adf43fa9f5f561785feda1d, 9a42068d36d24e25a3296e6a4eb8f918; no voters:
I20260808 14:38:52.773962 22331 raft_consensus.cc:2781] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:38:52.774024 22331 raft_consensus.cc:482] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:38:52.774052 22331 raft_consensus.cc:3037] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:52.775156 22331 raft_consensus.cc:504] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:52.775640 22331 leader_election.cc:288] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee359e24adf43fa9f5f561785feda1d (127.19.144.65:32785), 6d5bb3c8a0e741f98aad497ffa3e83ca (127.19.144.66:33193)
I20260808 14:38:52.776109 21959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2dcadc10634d0e8b47d2159dec4335" candidate_uuid: "9a42068d36d24e25a3296e6a4eb8f918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee359e24adf43fa9f5f561785feda1d"
I20260808 14:38:52.776279 21959 raft_consensus.cc:3037] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:52.776481 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2dcadc10634d0e8b47d2159dec4335" candidate_uuid: "9a42068d36d24e25a3296e6a4eb8f918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca"
I20260808 14:38:52.776647 22102 raft_consensus.cc:3037] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:52.777804 21959 raft_consensus.cc:2454] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a42068d36d24e25a3296e6a4eb8f918 in term 1.
I20260808 14:38:52.778012 22102 raft_consensus.cc:2454] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a42068d36d24e25a3296e6a4eb8f918 in term 1.
I20260808 14:38:52.778180 22175 leader_election.cc:302] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee359e24adf43fa9f5f561785feda1d, 9a42068d36d24e25a3296e6a4eb8f918; no voters:
I20260808 14:38:52.778458 22331 raft_consensus.cc:2781] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:38:52.778723 22331 raft_consensus.cc:686] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 LEADER]: Becoming Leader. State: Replica: 9a42068d36d24e25a3296e6a4eb8f918, State: Running, Role: LEADER
I20260808 14:38:52.779173 22331 consensus_queue.cc:237] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:52.781909 22346 tablet_replica.cc:439] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918: TxnStatusTablet state changed. Reason: New leader 9a42068d36d24e25a3296e6a4eb8f918. Latest consensus state: current_term: 1 leader_uuid: "9a42068d36d24e25a3296e6a4eb8f918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } }
I20260808 14:38:52.782124 22346 tablet_replica.cc:442] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:38:52.782459 22348 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 14:38:52.782696 22348 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 14:38:52.783043 21790 catalog_manager.cc:5495] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a42068d36d24e25a3296e6a4eb8f918 (127.19.144.67). New cstate: current_term: 1 leader_uuid: "9a42068d36d24e25a3296e6a4eb8f918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } health_report { overall_health: HEALTHY } } }
I20260808 14:38:52.797411 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.793185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:34289 (local address 127.0.0.1:56850)
0808 14:38:52.793356 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:52.793374 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:52.793497 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:52.793941 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:52.793944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:52.793952 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:38:52.794068 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.794074 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.794816 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.794820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.795536 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.795546 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.796069 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.796073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.796184 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.796248 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:38:52.797056 (+ 808us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:38:52.797076 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 14:38:52.797146 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 14:38:52.797263 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 14:38:52.797411 22320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.793260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56850 (local address 127.19.144.67:34289)
0808 14:38:52.793391 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.793398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.793431 (+ 33us) server_negotiation.cc:407] Connection header received
0808 14:38:52.793720 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.793723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.793783 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.793895 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:38:52.794179 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.794696 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.795657 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.795950 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.796065 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.796457 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:38:52.796960 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:38:52.797083 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.797167 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.797246 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:38:52.804215 21959 raft_consensus.cc:1261] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d [term 1 FOLLOWER]: Refusing update from remote peer 9a42068d36d24e25a3296e6a4eb8f918: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:38:52.804288 22102 raft_consensus.cc:1261] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 FOLLOWER]: Refusing update from remote peer 9a42068d36d24e25a3296e6a4eb8f918: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:38:52.804984 22331 consensus_queue.cc:1035] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:38:52.805163 22346 consensus_queue.cc:1035] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:38:52.808595 22332 tablet_replica.cc:439] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca: TxnStatusTablet state changed. Reason: New leader 9a42068d36d24e25a3296e6a4eb8f918. Latest consensus state: current_term: 1 leader_uuid: "9a42068d36d24e25a3296e6a4eb8f918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } }
I20260808 14:38:52.808736 22332 tablet_replica.cc:442] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:38:52.810920 22330 tablet_replica.cc:439] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d: TxnStatusTablet state changed. Reason: New leader 9a42068d36d24e25a3296e6a4eb8f918. Latest consensus state: current_term: 1 leader_uuid: "9a42068d36d24e25a3296e6a4eb8f918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } }
I20260808 14:38:52.811056 22330 tablet_replica.cc:442] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:38:52.813339 22346 tablet_replica.cc:439] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9a42068d36d24e25a3296e6a4eb8f918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } }
I20260808 14:38:52.813510 22346 tablet_replica.cc:442] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:38:52.813664 22330 tablet_replica.cc:439] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9a42068d36d24e25a3296e6a4eb8f918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } }
I20260808 14:38:52.813794 22330 tablet_replica.cc:442] T 4b2dcadc10634d0e8b47d2159dec4335 P 0ee359e24adf43fa9f5f561785feda1d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:38:52.817869 22354 mvcc.cc:205] Tried to move back new op lower bound from 7316274924757954560 to 7316274924666572800. Current Snapshot: MvccSnapshot[applied={T|T < 7316274924757954560}]
I20260808 14:38:52.823530 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.819575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34274 (local address 127.19.144.66:33193)
0808 14:38:52.819692 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.819697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.819825 (+ 128us) server_negotiation.cc:407] Connection header received
0808 14:38:52.820030 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.820033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.820102 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.820215 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:38:52.820490 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.821017 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.822069 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.822301 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.822411 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.822693 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:38:52.823135 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:38:52.823266 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.823329 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.823383 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 14:38:52.825539 22331 tablet_replica.cc:439] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9a42068d36d24e25a3296e6a4eb8f918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } }
I20260808 14:38:52.825727 22331 tablet_replica.cc:442] T 4b2dcadc10634d0e8b47d2159dec4335 P 9a42068d36d24e25a3296e6a4eb8f918: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:38:52.828405 22332 tablet_replica.cc:439] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9a42068d36d24e25a3296e6a4eb8f918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } } }
I20260808 14:38:52.828547 22332 tablet_replica.cc:442] T 4b2dcadc10634d0e8b47d2159dec4335 P 6d5bb3c8a0e741f98aad497ffa3e83ca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:38:52.836225 22363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.832122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:34289 (local address 127.19.144.66:43365)
0808 14:38:52.832517 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:52.832530 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:52.832624 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:52.832909 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:52.832911 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:52.832918 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:38:52.833064 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.833068 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.834251 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.834253 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.834869 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.834878 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.835500 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.835503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.835572 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.835619 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:38:52.835966 (+ 347us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:38:52.835972 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 14:38:52.836024 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 14:38:52.836089 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":123,"threads_started":1}
I20260808 14:38:52.836230 22320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.832211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:43365 (local address 127.19.144.67:34289)
0808 14:38:52.832361 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.832367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.832611 (+ 244us) server_negotiation.cc:407] Connection header received
0808 14:38:52.832728 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.832732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.832792 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.832886 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:38:52.833173 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.834142 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.834987 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.835364 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.835472 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.835782 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:38:52.835876 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:38:52.835958 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.836040 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.836096 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 14:38:52.843863 21959 raft_consensus.cc:1261] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d [term 1 FOLLOWER]: Refusing update from remote peer 6d5bb3c8a0e741f98aad497ffa3e83ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:38:52.844172 22252 raft_consensus.cc:1261] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Refusing update from remote peer 6d5bb3c8a0e741f98aad497ffa3e83ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:38:52.844400 22334 consensus_queue.cc:1035] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:38:52.844693 22332 consensus_queue.cc:1035] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:38:52.848337 22352 mvcc.cc:205] Tried to move back new op lower bound from 7316274924923424768 to 7316274923583418368. Current Snapshot: MvccSnapshot[applied={T|T < 7316274924923424768}]
I20260808 14:38:52.850982 22351 mvcc.cc:205] Tried to move back new op lower bound from 7316274924923424768 to 7316274923583418368. Current Snapshot: MvccSnapshot[applied={T|T < 7316274924923424768}]
I20260808 14:38:52.872059 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.868127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:46373 (local address 127.19.144.66:33193)
0808 14:38:52.868234 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:52.868241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:52.868288 (+ 47us) server_negotiation.cc:407] Connection header received
0808 14:38:52.868471 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:52.868474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:52.868538 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:52.868625 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:38:52.868907 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.869423 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.870631 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.871030 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.871149 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.871577 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:38:52.871659 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:38:52.871722 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:52.871858 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 14:38:52.871939 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 14:38:52.872100 22306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:52.868037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:33193 (local address 127.19.144.67:46373)
0808 14:38:52.868225 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:52.868241 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:52.868343 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:52.868634 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:52.868637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:52.868645 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:38:52.868768 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.868782 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.869539 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.869545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.870488 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:52.870499 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:52.871240 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:52.871246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:52.871361 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:52.871426 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:38:52.871756 (+ 330us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:38:52.871763 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 14:38:52.871853 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 14:38:52.871946 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 14:38:52.912123 21792 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:38:52.912376 21792 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:38:52.916038 21792 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c: Sending DeleteTablet for 3 replicas of tablet 040c68e49ec34a3ea19e7b76bd550977
I20260808 14:38:52.916953 22227 tablet_service.cc:1514] Processing DeleteTablet for tablet 040c68e49ec34a3ea19e7b76bd550977 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:52 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:56840
I20260808 14:38:52.917048 21933 tablet_service.cc:1514] Processing DeleteTablet for tablet 040c68e49ec34a3ea19e7b76bd550977 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:52 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:52656
I20260808 14:38:52.917644 22371 tablet_replica.cc:330] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d: stopping tablet replica
I20260808 14:38:52.917573 22081 tablet_service.cc:1514] Processing DeleteTablet for tablet 040c68e49ec34a3ea19e7b76bd550977 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:52 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:34262
I20260808 14:38:52.918036 22371 raft_consensus.cc:2229] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:38:52.918337 22371 raft_consensus.cc:2258] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:38:52.918262 21790 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 14:38:52.918787 22372 tablet_replica.cc:330] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca: stopping tablet replica
W20260808 14:38:52.918783 21790 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:38:52.919128 22372 raft_consensus.cc:2229] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:38:52.919590 22372 raft_consensus.cc:2258] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:38:52.921146 22372 ts_tablet_manager.cc:1905] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:52.921144 22371 ts_tablet_manager.cc:1905] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:52.925118 22081 tablet_service.cc:1467] Processing CreateTablet for tablet 5dcc585e36c048cc8d6f29039004b211 (DEFAULT_TABLE table=test-table [id=cf05fbd459d44a53b73ab06e9b61a3d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:52.925243 22227 tablet_service.cc:1467] Processing CreateTablet for tablet 5dcc585e36c048cc8d6f29039004b211 (DEFAULT_TABLE table=test-table [id=cf05fbd459d44a53b73ab06e9b61a3d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:52.925474 22081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dcc585e36c048cc8d6f29039004b211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:52.925544 22227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dcc585e36c048cc8d6f29039004b211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:52.928171 22325 tablet_bootstrap.cc:492] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918: Bootstrap starting.
I20260808 14:38:52.928493 22370 tablet_replica.cc:330] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918: stopping tablet replica
I20260808 14:38:52.929780 21933 tablet_service.cc:1467] Processing CreateTablet for tablet 5dcc585e36c048cc8d6f29039004b211 (DEFAULT_TABLE table=test-table [id=cf05fbd459d44a53b73ab06e9b61a3d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:52.929872 22371 ts_tablet_manager.cc:1918] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:38:52.929986 22371 log.cc:1196] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/wal/wals/040c68e49ec34a3ea19e7b76bd550977
I20260808 14:38:52.930101 21933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dcc585e36c048cc8d6f29039004b211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:52.930342 22371 ts_tablet_manager.cc:1939] T 040c68e49ec34a3ea19e7b76bd550977 P 0ee359e24adf43fa9f5f561785feda1d: Deleting consensus metadata
I20260808 14:38:52.930475 22370 raft_consensus.cc:2229] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:38:52.930678 22370 raft_consensus.cc:2258] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:38:52.931950 21775 catalog_manager.cc:4841] TS 0ee359e24adf43fa9f5f561785feda1d (127.19.144.65:32785): tablet 040c68e49ec34a3ea19e7b76bd550977 (table test-table [id=ca9a22b1d9324996a4398aa16a8097a9]) successfully deleted
I20260808 14:38:52.929435 22325 tablet_bootstrap.cc:654] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:52.933372 22324 tablet_bootstrap.cc:492] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d: Bootstrap starting.
I20260808 14:38:52.934449 22370 ts_tablet_manager.cc:1905] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:52.934527 22324 tablet_bootstrap.cc:654] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:52.935750 22326 tablet_bootstrap.cc:492] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Bootstrap starting.
I20260808 14:38:52.936946 22326 tablet_bootstrap.cc:654] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:52.938798 22372 ts_tablet_manager.cc:1918] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:38:52.938934 22372 log.cc:1196] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/040c68e49ec34a3ea19e7b76bd550977
I20260808 14:38:52.939358 22372 ts_tablet_manager.cc:1939] T 040c68e49ec34a3ea19e7b76bd550977 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Deleting consensus metadata
I20260808 14:38:52.940454 22370 ts_tablet_manager.cc:1918] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:38:52.940763 22370 log.cc:1196] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/040c68e49ec34a3ea19e7b76bd550977
I20260808 14:38:52.940826 21777 catalog_manager.cc:4841] TS 6d5bb3c8a0e741f98aad497ffa3e83ca (127.19.144.66:33193): tablet 040c68e49ec34a3ea19e7b76bd550977 (table test-table [id=ca9a22b1d9324996a4398aa16a8097a9]) successfully deleted
I20260808 14:38:52.942155 22324 tablet_bootstrap.cc:492] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d: No bootstrap required, opened a new log
I20260808 14:38:52.942329 22324 ts_tablet_manager.cc:1397] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260808 14:38:52.942612 22370 ts_tablet_manager.cc:1939] T 040c68e49ec34a3ea19e7b76bd550977 P 9a42068d36d24e25a3296e6a4eb8f918: Deleting consensus metadata
I20260808 14:38:52.942958 22324 raft_consensus.cc:348] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:52.943111 22324 raft_consensus.cc:374] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:52.943157 22324 raft_consensus.cc:729] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee359e24adf43fa9f5f561785feda1d, State: Initialized, Role: FOLLOWER
I20260808 14:38:52.943318 22324 consensus_queue.cc:260] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:52.943713 22324 ts_tablet_manager.cc:1428] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:38:52.944784 22325 tablet_bootstrap.cc:492] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918: No bootstrap required, opened a new log
I20260808 14:38:52.944898 22325 ts_tablet_manager.cc:1397] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20260808 14:38:52.945489 22325 raft_consensus.cc:348] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:52.945647 22325 raft_consensus.cc:374] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:52.945694 22325 raft_consensus.cc:729] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a42068d36d24e25a3296e6a4eb8f918, State: Initialized, Role: FOLLOWER
I20260808 14:38:52.945854 22325 consensus_queue.cc:260] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:52.946177 22325 ts_tablet_manager.cc:1428] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260808 14:38:52.949272 21776 catalog_manager.cc:4841] TS 9a42068d36d24e25a3296e6a4eb8f918 (127.19.144.67:34289): tablet 040c68e49ec34a3ea19e7b76bd550977 (table test-table [id=ca9a22b1d9324996a4398aa16a8097a9]) successfully deleted
I20260808 14:38:52.950093 22326 tablet_bootstrap.cc:492] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca: No bootstrap required, opened a new log
I20260808 14:38:52.950249 22326 ts_tablet_manager.cc:1397] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260808 14:38:52.950764 22326 raft_consensus.cc:348] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:52.950946 22326 raft_consensus.cc:374] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:52.950992 22326 raft_consensus.cc:729] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5bb3c8a0e741f98aad497ffa3e83ca, State: Initialized, Role: FOLLOWER
I20260808 14:38:52.951148 22326 consensus_queue.cc:260] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:52.951496 22326 ts_tablet_manager.cc:1428] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260808 14:38:53.074066 22368 raft_consensus.cc:482] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:38:53.074255 22368 raft_consensus.cc:504] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:53.074816 22368 leader_election.cc:288] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee359e24adf43fa9f5f561785feda1d (127.19.144.65:32785), 9a42068d36d24e25a3296e6a4eb8f918 (127.19.144.67:34289)
I20260808 14:38:53.075325 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dcc585e36c048cc8d6f29039004b211" candidate_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a42068d36d24e25a3296e6a4eb8f918" is_pre_election: true
I20260808 14:38:53.075371 21958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dcc585e36c048cc8d6f29039004b211" candidate_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee359e24adf43fa9f5f561785feda1d" is_pre_election: true
I20260808 14:38:53.075516 22252 raft_consensus.cc:2454] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5bb3c8a0e741f98aad497ffa3e83ca in term 0.
I20260808 14:38:53.075599 21958 raft_consensus.cc:2454] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5bb3c8a0e741f98aad497ffa3e83ca in term 0.
I20260808 14:38:53.075945 22033 leader_election.cc:302] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5bb3c8a0e741f98aad497ffa3e83ca, 9a42068d36d24e25a3296e6a4eb8f918; no voters:
I20260808 14:38:53.076131 22368 raft_consensus.cc:2781] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:38:53.076184 22368 raft_consensus.cc:482] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:38:53.076231 22368 raft_consensus.cc:3037] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:53.077332 22368 raft_consensus.cc:504] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:53.077826 22368 leader_election.cc:288] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [CANDIDATE]: Term 1 election: Requested vote from peers 0ee359e24adf43fa9f5f561785feda1d (127.19.144.65:32785), 9a42068d36d24e25a3296e6a4eb8f918 (127.19.144.67:34289)
I20260808 14:38:53.078269 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dcc585e36c048cc8d6f29039004b211" candidate_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a42068d36d24e25a3296e6a4eb8f918"
I20260808 14:38:53.078305 21958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dcc585e36c048cc8d6f29039004b211" candidate_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee359e24adf43fa9f5f561785feda1d"
I20260808 14:38:53.078418 22252 raft_consensus.cc:3037] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:53.078449 21958 raft_consensus.cc:3037] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:53.079550 21958 raft_consensus.cc:2454] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5bb3c8a0e741f98aad497ffa3e83ca in term 1.
I20260808 14:38:53.079663 22252 raft_consensus.cc:2454] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5bb3c8a0e741f98aad497ffa3e83ca in term 1.
I20260808 14:38:53.079898 22032 leader_election.cc:302] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee359e24adf43fa9f5f561785feda1d, 6d5bb3c8a0e741f98aad497ffa3e83ca; no voters:
I20260808 14:38:53.080130 22368 raft_consensus.cc:2781] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:38:53.080257 22368 raft_consensus.cc:686] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 LEADER]: Becoming Leader. State: Replica: 6d5bb3c8a0e741f98aad497ffa3e83ca, State: Running, Role: LEADER
I20260808 14:38:53.080412 22368 consensus_queue.cc:237] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:53.082427 21792 catalog_manager.cc:5495] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d5bb3c8a0e741f98aad497ffa3e83ca (127.19.144.66). New cstate: current_term: 1 leader_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } health_report { overall_health: UNKNOWN } } }
I20260808 14:38:53.107910 21958 raft_consensus.cc:1261] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d [term 1 FOLLOWER]: Refusing update from remote peer 6d5bb3c8a0e741f98aad497ffa3e83ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:38:53.107985 22252 raft_consensus.cc:1261] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Refusing update from remote peer 6d5bb3c8a0e741f98aad497ffa3e83ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:38:53.108421 22332 consensus_queue.cc:1035] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:38:53.108582 22368 consensus_queue.cc:1035] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 14:38:53.125054 22352 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 14:38:54.687150 21791 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:38:54.687340 21791 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:38:54.689927 21791 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c: Sending DeleteTablet for 3 replicas of tablet 5dcc585e36c048cc8d6f29039004b211
I20260808 14:38:54.690590 21933 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dcc585e36c048cc8d6f29039004b211 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:54 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:52656
I20260808 14:38:54.690663 22226 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dcc585e36c048cc8d6f29039004b211 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:54 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:56840
I20260808 14:38:54.690704 22081 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dcc585e36c048cc8d6f29039004b211 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:54 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:34262
I20260808 14:38:54.691067 22402 tablet_replica.cc:330] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918: stopping tablet replica
I20260808 14:38:54.691229 22402 raft_consensus.cc:2229] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:38:54.691377 22402 raft_consensus.cc:2258] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:38:54.691651 22401 tablet_replica.cc:330] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d: stopping tablet replica
I20260808 14:38:54.691788 22401 raft_consensus.cc:2229] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:38:54.691965 22401 raft_consensus.cc:2258] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:38:54.692193 22402 ts_tablet_manager.cc:1905] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:54.692730 22401 ts_tablet_manager.cc:1905] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:54.694698 22403 tablet_replica.cc:330] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca: stopping tablet replica
I20260808 14:38:54.694911 22403 raft_consensus.cc:2229] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:38:54.695152 22403 raft_consensus.cc:2258] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:38:54.695747 22402 ts_tablet_manager.cc:1918] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 14:38:54.695842 22402 log.cc:1196] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/5dcc585e36c048cc8d6f29039004b211
I20260808 14:38:54.695818 22403 ts_tablet_manager.cc:1905] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:54.696141 22402 ts_tablet_manager.cc:1939] T 5dcc585e36c048cc8d6f29039004b211 P 9a42068d36d24e25a3296e6a4eb8f918: Deleting consensus metadata
I20260808 14:38:54.696326 22401 ts_tablet_manager.cc:1918] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 14:38:54.696496 22401 log.cc:1196] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/wal/wals/5dcc585e36c048cc8d6f29039004b211
I20260808 14:38:54.696847 22401 ts_tablet_manager.cc:1939] T 5dcc585e36c048cc8d6f29039004b211 P 0ee359e24adf43fa9f5f561785feda1d: Deleting consensus metadata
I20260808 14:38:54.697088 21776 catalog_manager.cc:4841] TS 9a42068d36d24e25a3296e6a4eb8f918 (127.19.144.67:34289): tablet 5dcc585e36c048cc8d6f29039004b211 (table test-table [id=cf05fbd459d44a53b73ab06e9b61a3d7]) successfully deleted
I20260808 14:38:54.697662 21775 catalog_manager.cc:4841] TS 0ee359e24adf43fa9f5f561785feda1d (127.19.144.65:32785): tablet 5dcc585e36c048cc8d6f29039004b211 (table test-table [id=cf05fbd459d44a53b73ab06e9b61a3d7]) successfully deleted
I20260808 14:38:54.698778 22403 ts_tablet_manager.cc:1918] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 14:38:54.698853 22403 log.cc:1196] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/5dcc585e36c048cc8d6f29039004b211
I20260808 14:38:54.699170 22403 ts_tablet_manager.cc:1939] T 5dcc585e36c048cc8d6f29039004b211 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Deleting consensus metadata
I20260808 14:38:54.699898 21777 catalog_manager.cc:4841] TS 6d5bb3c8a0e741f98aad497ffa3e83ca (127.19.144.66:33193): tablet 5dcc585e36c048cc8d6f29039004b211 (table test-table [id=cf05fbd459d44a53b73ab06e9b61a3d7]) successfully deleted
Aug 08 14:38:54 dist-test-slave-p3t5 krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199934, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 14:38:54 dist-test-slave-p3t5 krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199934, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:54.723053 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:54.713607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48574 (local address 127.19.144.126:45617)
0808 14:38:54.713990 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:54.713998 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:54.714014 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:38:54.714145 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:54.714147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:54.714192 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:54.714276 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:54.714990 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.715540 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.716321 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.716536 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.716619 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:54.719488 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:54.719491 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:54.719493 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:54.719522 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:54.721186 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:54.721675 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:54.721680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:54.721681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:54.721773 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:54.722079 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:54.722083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:54.722086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:54.722291 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:54.722453 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:54.722725 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 14:38:54.722864 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":121,"threads_started":1}
I20260808 14:38:54.725937 21791 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:38:54.726351 21791 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:38:54.731271 21933 tablet_service.cc:1467] Processing CreateTablet for tablet 4f98bec6d303460d84566fd7ec509bb7 (DEFAULT_TABLE table=test-table [id=a3b991a6103741f49cbd2913147ebb5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:54.731287 22081 tablet_service.cc:1467] Processing CreateTablet for tablet 4f98bec6d303460d84566fd7ec509bb7 (DEFAULT_TABLE table=test-table [id=a3b991a6103741f49cbd2913147ebb5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:54.731611 22081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f98bec6d303460d84566fd7ec509bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:54.731609 21933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f98bec6d303460d84566fd7ec509bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:54.731621 22226 tablet_service.cc:1467] Processing CreateTablet for tablet 4f98bec6d303460d84566fd7ec509bb7 (DEFAULT_TABLE table=test-table [id=a3b991a6103741f49cbd2913147ebb5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:54.731920 22226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f98bec6d303460d84566fd7ec509bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:54.734380 22415 tablet_bootstrap.cc:492] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918: Bootstrap starting.
I20260808 14:38:54.734445 22413 tablet_bootstrap.cc:492] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Bootstrap starting.
I20260808 14:38:54.734618 22414 tablet_bootstrap.cc:492] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d: Bootstrap starting.
I20260808 14:38:54.735368 22415 tablet_bootstrap.cc:654] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:54.735370 22413 tablet_bootstrap.cc:654] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:54.735438 22414 tablet_bootstrap.cc:654] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:54.736938 22414 tablet_bootstrap.cc:492] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d: No bootstrap required, opened a new log
I20260808 14:38:54.736979 22415 tablet_bootstrap.cc:492] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918: No bootstrap required, opened a new log
I20260808 14:38:54.737039 22414 ts_tablet_manager.cc:1397] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 14:38:54.737082 22415 ts_tablet_manager.cc:1397] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:38:54.737370 22413 tablet_bootstrap.cc:492] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca: No bootstrap required, opened a new log
I20260808 14:38:54.737470 22413 ts_tablet_manager.cc:1397] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:38:54.737558 22414 raft_consensus.cc:348] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.737673 22415 raft_consensus.cc:348] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.737740 22414 raft_consensus.cc:374] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:54.737784 22414 raft_consensus.cc:729] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee359e24adf43fa9f5f561785feda1d, State: Initialized, Role: FOLLOWER
I20260808 14:38:54.737790 22415 raft_consensus.cc:374] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:54.737828 22415 raft_consensus.cc:729] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a42068d36d24e25a3296e6a4eb8f918, State: Initialized, Role: FOLLOWER
I20260808 14:38:54.737910 22413 raft_consensus.cc:348] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.737936 22414 consensus_queue.cc:260] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.737972 22415 consensus_queue.cc:260] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.738026 22413 raft_consensus.cc:374] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:54.738060 22413 raft_consensus.cc:729] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5bb3c8a0e741f98aad497ffa3e83ca, State: Initialized, Role: FOLLOWER
I20260808 14:38:54.738185 22413 consensus_queue.cc:260] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.738263 22415 ts_tablet_manager.cc:1428] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:38:54.738492 22413 ts_tablet_manager.cc:1428] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 14:38:54.739548 22414 ts_tablet_manager.cc:1428] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 14:38:54.792712 22419 raft_consensus.cc:482] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:38:54.792886 22419 raft_consensus.cc:504] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.794293 22419 leader_election.cc:288] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d5bb3c8a0e741f98aad497ffa3e83ca (127.19.144.66:33193), 9a42068d36d24e25a3296e6a4eb8f918 (127.19.144.67:34289)
I20260808 14:38:54.799098 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:54.795081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:41109 (local address 127.19.144.66:33193)
0808 14:38:54.795384 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:54.795392 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:54.795405 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:38:54.795450 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:54.795453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:54.795493 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:54.795565 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:54.795924 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.796622 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.797990 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.798465 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.798567 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:54.798687 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:54.798849 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 14:38:54.798933 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":102,"threads_started":1}
I20260808 14:38:54.799122 22420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:54.794511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:33193 (local address 127.19.144.65:41109)
0808 14:38:54.795037 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:54.795055 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:54.795166 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:54.795603 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:54.795606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:54.795624 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:54.795775 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:54.795782 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.796732 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.796735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:54.797873 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:54.797883 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.798585 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.798590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:54.798697 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:54.798726 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 14:38:54.798811 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 14:38:54.798928 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":3,"thread_start_us":135,"threads_started":1}
I20260808 14:38:54.799342 22421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:54.794622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:34289 (local address 127.19.144.65:51253)
0808 14:38:54.795105 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:54.795121 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:54.795235 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:54.795754 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:54.795756 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:54.795771 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:38:54.795919 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:54.795926 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.796912 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.796915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:54.797982 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:54.797992 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.798854 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.798858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:54.798956 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:54.798980 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:38:54.799058 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 14:38:54.799130 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":22,"thread_start_us":125,"threads_started":1}
I20260808 14:38:54.799347 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:54.794778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:51253 (local address 127.19.144.67:34289)
0808 14:38:54.795484 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:54.795491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:54.795503 (+ 12us) server_negotiation.cc:407] Connection header received
0808 14:38:54.795571 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:54.795575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:54.795624 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:54.795716 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:38:54.796044 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.796758 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.798104 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.798703 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.798831 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:54.798878 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:54.799098 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 14:38:54.799192 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":119,"threads_started":1}
I20260808 14:38:54.799916 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f98bec6d303460d84566fd7ec509bb7" candidate_uuid: "0ee359e24adf43fa9f5f561785feda1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a42068d36d24e25a3296e6a4eb8f918" is_pre_election: true
I20260808 14:38:54.799975 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f98bec6d303460d84566fd7ec509bb7" candidate_uuid: "0ee359e24adf43fa9f5f561785feda1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" is_pre_election: true
I20260808 14:38:54.800098 22252 raft_consensus.cc:2454] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee359e24adf43fa9f5f561785feda1d in term 0.
I20260808 14:38:54.800125 22102 raft_consensus.cc:2454] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee359e24adf43fa9f5f561785feda1d in term 0.
I20260808 14:38:54.800539 21885 leader_election.cc:302] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0ee359e24adf43fa9f5f561785feda1d, 6d5bb3c8a0e741f98aad497ffa3e83ca, 9a42068d36d24e25a3296e6a4eb8f918; no voters:
I20260808 14:38:54.800817 22419 raft_consensus.cc:2781] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:38:54.800882 22419 raft_consensus.cc:482] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:38:54.800921 22419 raft_consensus.cc:3037] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:54.802095 22419 raft_consensus.cc:504] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.802537 22419 leader_election.cc:288] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [CANDIDATE]: Term 1 election: Requested vote from peers 6d5bb3c8a0e741f98aad497ffa3e83ca (127.19.144.66:33193), 9a42068d36d24e25a3296e6a4eb8f918 (127.19.144.67:34289)
I20260808 14:38:54.802904 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f98bec6d303460d84566fd7ec509bb7" candidate_uuid: "0ee359e24adf43fa9f5f561785feda1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca"
I20260808 14:38:54.803016 22102 raft_consensus.cc:3037] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:54.803006 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f98bec6d303460d84566fd7ec509bb7" candidate_uuid: "0ee359e24adf43fa9f5f561785feda1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a42068d36d24e25a3296e6a4eb8f918"
I20260808 14:38:54.803143 22252 raft_consensus.cc:3037] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:54.804040 22102 raft_consensus.cc:2454] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee359e24adf43fa9f5f561785feda1d in term 1.
I20260808 14:38:54.804157 22252 raft_consensus.cc:2454] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee359e24adf43fa9f5f561785feda1d in term 1.
I20260808 14:38:54.804545 21885 leader_election.cc:302] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0ee359e24adf43fa9f5f561785feda1d, 6d5bb3c8a0e741f98aad497ffa3e83ca, 9a42068d36d24e25a3296e6a4eb8f918; no voters:
I20260808 14:38:54.804744 22419 raft_consensus.cc:2781] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:38:54.804976 22419 raft_consensus.cc:686] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [term 1 LEADER]: Becoming Leader. State: Replica: 0ee359e24adf43fa9f5f561785feda1d, State: Running, Role: LEADER
I20260808 14:38:54.805264 22419 consensus_queue.cc:237] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.807819 21790 catalog_manager.cc:5495] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee359e24adf43fa9f5f561785feda1d (127.19.144.65). New cstate: current_term: 1 leader_uuid: "0ee359e24adf43fa9f5f561785feda1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } health_report { overall_health: UNKNOWN } } }
I20260808 14:38:54.824577 22427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:54.820666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52666 (local address 127.19.144.65:32785)
0808 14:38:54.820983 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:54.820990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:54.821002 (+ 12us) server_negotiation.cc:407] Connection header received
0808 14:38:54.821079 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:54.821081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:54.821135 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:54.821243 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:38:54.821494 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.822119 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.822982 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.823213 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.823322 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:54.823619 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:38:54.824106 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:38:54.824197 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:54.824285 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 14:38:54.824338 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":146,"threads_started":1}
I20260808 14:38:54.835670 22430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:54.832172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:34289 (local address 127.19.144.65:46919)
0808 14:38:54.832515 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:54.832528 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:54.832626 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:54.832849 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:54.832852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:54.832858 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:38:54.832976 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:54.832980 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.833807 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.833810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:54.834457 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:54.834467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.834955 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.834958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:54.835029 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:54.835076 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:38:54.835388 (+ 312us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:38:54.835394 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 14:38:54.835447 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 14:38:54.835518 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":101,"threads_started":1}
I20260808 14:38:54.835693 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:54.832337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:46919 (local address 127.19.144.67:34289)
0808 14:38:54.832508 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:54.832511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:54.832598 (+ 87us) server_negotiation.cc:407] Connection header received
0808 14:38:54.832714 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:54.832716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:54.832761 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:54.832828 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:38:54.833127 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.833698 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.834574 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.834813 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.834901 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:54.835221 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:38:54.835300 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:38:54.835361 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:54.835513 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 14:38:54.835584 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 14:38:54.842792 22102 raft_consensus.cc:1261] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 FOLLOWER]: Refusing update from remote peer 0ee359e24adf43fa9f5f561785feda1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:38:54.842871 22252 raft_consensus.cc:1261] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Refusing update from remote peer 0ee359e24adf43fa9f5f561785feda1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:38:54.843355 22424 consensus_queue.cc:1035] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:38:54.843566 22419 consensus_queue.cc:1035] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:38:54.861581 22427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:54.858088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:43415 (local address 127.19.144.65:32785)
0808 14:38:54.858214 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:54.858218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:54.858416 (+ 198us) server_negotiation.cc:407] Connection header received
0808 14:38:54.858573 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:54.858576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:54.858632 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:54.858715 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:38:54.858971 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.859506 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.860457 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.860688 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.860804 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:54.861068 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:38:54.861151 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:38:54.861233 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:54.861369 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 14:38:54.861425 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 14:38:54.861584 22435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:54.858022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:32785 (local address 127.19.144.67:43415)
0808 14:38:54.858322 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:54.858335 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:54.858460 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:54.858729 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:54.858732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:54.858741 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:38:54.858854 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:54.858860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.859631 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.859634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:54.860296 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:54.860306 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.860805 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.860809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:54.860898 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:54.860956 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:38:54.861248 (+ 292us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:38:54.861255 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 14:38:54.861353 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 14:38:54.861442 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":122,"threads_started":1}
I20260808 14:38:54.887130 21790 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:38:54.887336 21790 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:38:54.889875 21790 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c: Sending DeleteTablet for 3 replicas of tablet 4f98bec6d303460d84566fd7ec509bb7
I20260808 14:38:54.890647 22081 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f98bec6d303460d84566fd7ec509bb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:54 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:34262
I20260808 14:38:54.890882 22403 tablet_replica.cc:330] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca: stopping tablet replica
I20260808 14:38:54.891038 22403 raft_consensus.cc:2229] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:38:54.891175 22403 raft_consensus.cc:2258] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:38:54.891683 22226 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f98bec6d303460d84566fd7ec509bb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:54 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:56840
I20260808 14:38:54.891885 22402 tablet_replica.cc:330] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918: stopping tablet replica
I20260808 14:38:54.892001 22402 raft_consensus.cc:2229] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:38:54.892123 22402 raft_consensus.cc:2258] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:38:54.892411 22403 ts_tablet_manager.cc:1905] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:54.894714 22403 ts_tablet_manager.cc:1918] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:38:54.894821 22403 log.cc:1196] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/4f98bec6d303460d84566fd7ec509bb7
I20260808 14:38:54.895112 22403 ts_tablet_manager.cc:1939] T 4f98bec6d303460d84566fd7ec509bb7 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Deleting consensus metadata
I20260808 14:38:54.895467 21933 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f98bec6d303460d84566fd7ec509bb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:54 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:52656
I20260808 14:38:54.895690 22401 tablet_replica.cc:330] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d: stopping tablet replica
I20260808 14:38:54.895860 22401 raft_consensus.cc:2229] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:38:54.896291 22401 raft_consensus.cc:2258] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:38:54.896324 22402 ts_tablet_manager.cc:1905] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:54.897578 22401 ts_tablet_manager.cc:1905] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:54.897558 21790 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:38:54.898113 21790 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:38:54.899070 22402 ts_tablet_manager.cc:1918] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:38:54.899153 22402 log.cc:1196] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/4f98bec6d303460d84566fd7ec509bb7
I20260808 14:38:54.899421 22402 ts_tablet_manager.cc:1939] T 4f98bec6d303460d84566fd7ec509bb7 P 9a42068d36d24e25a3296e6a4eb8f918: Deleting consensus metadata
I20260808 14:38:54.900338 21776 catalog_manager.cc:4841] TS 9a42068d36d24e25a3296e6a4eb8f918 (127.19.144.67:34289): tablet 4f98bec6d303460d84566fd7ec509bb7 (table test-table [id=a3b991a6103741f49cbd2913147ebb5e]) successfully deleted
I20260808 14:38:54.900379 22401 ts_tablet_manager.cc:1918] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:38:54.900447 22401 log.cc:1196] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-0/wal/wals/4f98bec6d303460d84566fd7ec509bb7
I20260808 14:38:54.900712 22401 ts_tablet_manager.cc:1939] T 4f98bec6d303460d84566fd7ec509bb7 P 0ee359e24adf43fa9f5f561785feda1d: Deleting consensus metadata
I20260808 14:38:54.901477 21775 catalog_manager.cc:4841] TS 0ee359e24adf43fa9f5f561785feda1d (127.19.144.65:32785): tablet 4f98bec6d303460d84566fd7ec509bb7 (table test-table [id=a3b991a6103741f49cbd2913147ebb5e]) successfully deleted
I20260808 14:38:54.902792 21777 catalog_manager.cc:4841] TS 6d5bb3c8a0e741f98aad497ffa3e83ca (127.19.144.66:33193): tablet 4f98bec6d303460d84566fd7ec509bb7 (table test-table [id=a3b991a6103741f49cbd2913147ebb5e]) successfully deleted
I20260808 14:38:54.906235 21933 tablet_service.cc:1467] Processing CreateTablet for tablet 554a3cf99fa64132b49a8b85141438d9 (DEFAULT_TABLE table=test-table [id=acf3ea994c4f46d292e06212396571f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:54.906352 22226 tablet_service.cc:1467] Processing CreateTablet for tablet 554a3cf99fa64132b49a8b85141438d9 (DEFAULT_TABLE table=test-table [id=acf3ea994c4f46d292e06212396571f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:54.906527 21933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 554a3cf99fa64132b49a8b85141438d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:54.906466 22081 tablet_service.cc:1467] Processing CreateTablet for tablet 554a3cf99fa64132b49a8b85141438d9 (DEFAULT_TABLE table=test-table [id=acf3ea994c4f46d292e06212396571f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:38:54.906657 22226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 554a3cf99fa64132b49a8b85141438d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:54.906737 22081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 554a3cf99fa64132b49a8b85141438d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:54.908941 22414 tablet_bootstrap.cc:492] T 554a3cf99fa64132b49a8b85141438d9 P 0ee359e24adf43fa9f5f561785feda1d: Bootstrap starting.
I20260808 14:38:54.909035 22415 tablet_bootstrap.cc:492] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918: Bootstrap starting.
I20260808 14:38:54.909976 22414 tablet_bootstrap.cc:654] T 554a3cf99fa64132b49a8b85141438d9 P 0ee359e24adf43fa9f5f561785feda1d: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:54.909976 22415 tablet_bootstrap.cc:654] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:54.910624 22413 tablet_bootstrap.cc:492] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Bootstrap starting.
I20260808 14:38:54.911533 22413 tablet_bootstrap.cc:654] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:54.911794 22415 tablet_bootstrap.cc:492] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918: No bootstrap required, opened a new log
I20260808 14:38:54.911911 22415 ts_tablet_manager.cc:1397] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:38:54.912354 22415 raft_consensus.cc:348] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.912508 22415 raft_consensus.cc:374] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:54.912549 22415 raft_consensus.cc:729] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a42068d36d24e25a3296e6a4eb8f918, State: Initialized, Role: FOLLOWER
I20260808 14:38:54.912576 22414 tablet_bootstrap.cc:492] T 554a3cf99fa64132b49a8b85141438d9 P 0ee359e24adf43fa9f5f561785feda1d: No bootstrap required, opened a new log
I20260808 14:38:54.912658 22414 ts_tablet_manager.cc:1397] T 554a3cf99fa64132b49a8b85141438d9 P 0ee359e24adf43fa9f5f561785feda1d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 14:38:54.912685 22415 consensus_queue.cc:260] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.912966 22415 ts_tablet_manager.cc:1428] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 14:38:54.913153 22414 raft_consensus.cc:348] T 554a3cf99fa64132b49a8b85141438d9 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.913285 22414 raft_consensus.cc:374] T 554a3cf99fa64132b49a8b85141438d9 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:54.913318 22414 raft_consensus.cc:729] T 554a3cf99fa64132b49a8b85141438d9 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee359e24adf43fa9f5f561785feda1d, State: Initialized, Role: FOLLOWER
I20260808 14:38:54.913460 22414 consensus_queue.cc:260] T 554a3cf99fa64132b49a8b85141438d9 P 0ee359e24adf43fa9f5f561785feda1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.913774 22414 ts_tablet_manager.cc:1428] T 554a3cf99fa64132b49a8b85141438d9 P 0ee359e24adf43fa9f5f561785feda1d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:38:54.918602 22413 tablet_bootstrap.cc:492] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca: No bootstrap required, opened a new log
I20260808 14:38:54.918744 22413 ts_tablet_manager.cc:1397] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260808 14:38:54.919173 22413 raft_consensus.cc:348] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.919309 22413 raft_consensus.cc:374] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:54.919351 22413 raft_consensus.cc:729] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5bb3c8a0e741f98aad497ffa3e83ca, State: Initialized, Role: FOLLOWER
I20260808 14:38:54.919495 22413 consensus_queue.cc:260] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.919833 22413 ts_tablet_manager.cc:1428] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 14:38:54.927616 22388 raft_consensus.cc:482] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:38:54.927772 22388 raft_consensus.cc:504] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.928241 22388 leader_election.cc:288] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee359e24adf43fa9f5f561785feda1d (127.19.144.65:32785), 6d5bb3c8a0e741f98aad497ffa3e83ca (127.19.144.66:33193)
I20260808 14:38:54.928676 21958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "554a3cf99fa64132b49a8b85141438d9" candidate_uuid: "9a42068d36d24e25a3296e6a4eb8f918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee359e24adf43fa9f5f561785feda1d" is_pre_election: true
I20260808 14:38:54.928685 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "554a3cf99fa64132b49a8b85141438d9" candidate_uuid: "9a42068d36d24e25a3296e6a4eb8f918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" is_pre_election: true
I20260808 14:38:54.928854 21958 raft_consensus.cc:2454] T 554a3cf99fa64132b49a8b85141438d9 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a42068d36d24e25a3296e6a4eb8f918 in term 0.
I20260808 14:38:54.928855 22102 raft_consensus.cc:2454] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a42068d36d24e25a3296e6a4eb8f918 in term 0.
I20260808 14:38:54.929167 22177 leader_election.cc:302] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5bb3c8a0e741f98aad497ffa3e83ca, 9a42068d36d24e25a3296e6a4eb8f918; no voters:
I20260808 14:38:54.929584 22388 raft_consensus.cc:2781] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:38:54.929694 22388 raft_consensus.cc:482] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:38:54.929741 22388 raft_consensus.cc:3037] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:54.931012 22388 raft_consensus.cc:504] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.931382 22388 leader_election.cc:288] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee359e24adf43fa9f5f561785feda1d (127.19.144.65:32785), 6d5bb3c8a0e741f98aad497ffa3e83ca (127.19.144.66:33193)
I20260808 14:38:54.931772 21958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "554a3cf99fa64132b49a8b85141438d9" candidate_uuid: "9a42068d36d24e25a3296e6a4eb8f918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee359e24adf43fa9f5f561785feda1d"
I20260808 14:38:54.931913 21958 raft_consensus.cc:3037] T 554a3cf99fa64132b49a8b85141438d9 P 0ee359e24adf43fa9f5f561785feda1d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:54.931957 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "554a3cf99fa64132b49a8b85141438d9" candidate_uuid: "9a42068d36d24e25a3296e6a4eb8f918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca"
I20260808 14:38:54.932087 22102 raft_consensus.cc:3037] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:54.932922 21958 raft_consensus.cc:2454] T 554a3cf99fa64132b49a8b85141438d9 P 0ee359e24adf43fa9f5f561785feda1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a42068d36d24e25a3296e6a4eb8f918 in term 1.
I20260808 14:38:54.933204 22102 raft_consensus.cc:2454] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a42068d36d24e25a3296e6a4eb8f918 in term 1.
I20260808 14:38:54.933228 22175 leader_election.cc:302] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee359e24adf43fa9f5f561785feda1d, 9a42068d36d24e25a3296e6a4eb8f918; no voters:
I20260808 14:38:54.933445 22388 raft_consensus.cc:2781] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:38:54.933565 22388 raft_consensus.cc:686] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 LEADER]: Becoming Leader. State: Replica: 9a42068d36d24e25a3296e6a4eb8f918, State: Running, Role: LEADER
I20260808 14:38:54.933763 22388 consensus_queue.cc:237] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } }
I20260808 14:38:54.935688 21790 catalog_manager.cc:5495] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a42068d36d24e25a3296e6a4eb8f918 (127.19.144.67). New cstate: current_term: 1 leader_uuid: "9a42068d36d24e25a3296e6a4eb8f918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a42068d36d24e25a3296e6a4eb8f918" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34289 } health_report { overall_health: HEALTHY } } }
I20260808 14:38:54.956883 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:54.953287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56862 (local address 127.19.144.67:34289)
0808 14:38:54.953429 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:54.953433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:54.953446 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:38:54.953579 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:54.953582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:54.953683 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:54.953757 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:38:54.954046 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.954571 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.955759 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.955997 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.956089 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:54.956366 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:38:54.956437 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:38:54.956494 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:54.956613 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 14:38:54.956660 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 14:38:54.967036 22435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:54.963673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:34289 (local address 127.19.144.67:59697)
0808 14:38:54.963889 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:54.963907 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:54.964001 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:54.964272 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:54.964275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:54.964282 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:38:54.964413 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:54.964420 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.965157 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.965161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:54.965873 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:54.965884 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.966323 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.966327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:54.966407 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:54.966440 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:38:54.966738 (+ 298us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:38:54.966746 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 14:38:54.966809 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 14:38:54.966899 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":26}
I20260808 14:38:54.967036 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:54.963833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:59697 (local address 127.19.144.67:34289)
0808 14:38:54.963954 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:54.963959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:54.963972 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:38:54.964100 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:54.964103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:54.964150 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:54.964242 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:38:54.964522 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.965036 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.965994 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:54.966211 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:54.966322 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:54.966559 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:38:54.966638 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:38:54.966733 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:54.966826 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 14:38:54.966872 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":48}
I20260808 14:38:54.973418 21958 raft_consensus.cc:1261] T 554a3cf99fa64132b49a8b85141438d9 P 0ee359e24adf43fa9f5f561785feda1d [term 1 FOLLOWER]: Refusing update from remote peer 9a42068d36d24e25a3296e6a4eb8f918: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:38:54.973752 22102 raft_consensus.cc:1261] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 FOLLOWER]: Refusing update from remote peer 9a42068d36d24e25a3296e6a4eb8f918: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:38:54.973845 22433 consensus_queue.cc:1035] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee359e24adf43fa9f5f561785feda1d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 32785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:38:54.974195 22433 consensus_queue.cc:1035] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5bb3c8a0e741f98aad497ffa3e83ca" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 14:38:54.990942 22380 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 14:38:56.555907 21792 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:38:56.556071 21792 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:38:56.559401 21792 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c14732906c34c74bf7e5044becee68c: Sending DeleteTablet for 3 replicas of tablet 554a3cf99fa64132b49a8b85141438d9
I20260808 14:38:56.560132 22081 tablet_service.cc:1514] Processing DeleteTablet for tablet 554a3cf99fa64132b49a8b85141438d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:56 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:34262
I20260808 14:38:56.560237 22226 tablet_service.cc:1514] Processing DeleteTablet for tablet 554a3cf99fa64132b49a8b85141438d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:56 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:56840
I20260808 14:38:56.560618 22465 tablet_replica.cc:330] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca: stopping tablet replica
I20260808 14:38:56.560778 22465 raft_consensus.cc:2229] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:38:56.560935 22465 raft_consensus.cc:2258] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:38:56.561178 22466 tablet_replica.cc:330] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918: stopping tablet replica
I20260808 14:38:56.561386 22466 raft_consensus.cc:2229] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:38:56.561800 22466 raft_consensus.cc:2258] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:38:56.561761 21933 tablet_service.cc:1514] Processing DeleteTablet for tablet 554a3cf99fa64132b49a8b85141438d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:38:56 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:52656
I20260808 14:38:56.562443 22465 ts_tablet_manager.cc:1905] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:56.562582 22467 tablet_replica.cc:330] T 554a3cf99fa64132b49a8b85141438d9 P 0ee359e24adf43fa9f5f561785feda1d: stopping tablet replica
I20260808 14:38:56.562645 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 21866
I20260808 14:38:56.564873 22466 ts_tablet_manager.cc:1905] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:38:56.568084 22466 ts_tablet_manager.cc:1918] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 14:38:56.568179 22466 log.cc:1196] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/554a3cf99fa64132b49a8b85141438d9
I20260808 14:38:56.568507 22466 ts_tablet_manager.cc:1939] T 554a3cf99fa64132b49a8b85141438d9 P 9a42068d36d24e25a3296e6a4eb8f918: Deleting consensus metadata
I20260808 14:38:56.569298 21776 catalog_manager.cc:4841] TS 9a42068d36d24e25a3296e6a4eb8f918 (127.19.144.67:34289): tablet 554a3cf99fa64132b49a8b85141438d9 (table test-table [id=acf3ea994c4f46d292e06212396571f8]) successfully deleted
I20260808 14:38:56.571385 22465 ts_tablet_manager.cc:1918] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 14:38:56.571486 22465 log.cc:1196] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/554a3cf99fa64132b49a8b85141438d9
I20260808 14:38:56.571733 22465 ts_tablet_manager.cc:1939] T 554a3cf99fa64132b49a8b85141438d9 P 6d5bb3c8a0e741f98aad497ffa3e83ca: Deleting consensus metadata
I20260808 14:38:56.572467 21777 catalog_manager.cc:4841] TS 6d5bb3c8a0e741f98aad497ffa3e83ca (127.19.144.66:33193): tablet 554a3cf99fa64132b49a8b85141438d9 (table test-table [id=acf3ea994c4f46d292e06212396571f8]) successfully deleted
W20260808 14:38:56.575100 21775 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.144.65:32785 (error 108)
I20260808 14:38:56.576005 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 22016
I20260808 14:38:56.582578 22469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:56.575916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:32785 (local address 127.0.0.1:59364)
0808 14:38:56.582317 (+ 6401us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:56.582458 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:32785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":88,"threads_started":1}
W20260808 14:38:56.582744 21775 catalog_manager.cc:4584] TS 0ee359e24adf43fa9f5f561785feda1d (127.19.144.65:32785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 554a3cf99fa64132b49a8b85141438d9: Network error: Client connection negotiation failed: client connection to 127.19.144.65:32785: connect: Connection refused (error 111)
I20260808 14:38:56.587934 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 22159
I20260808 14:38:56.597275 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 21759
2026-08-08T14:38:56Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6471 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:38:56 dist-test-slave-p3t5 krb5kdc[22472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:38:56 dist-test-slave-p3t5 krb5kdc[22472](info): set up 2 sockets
Aug 08 14:38:56 dist-test-slave-p3t5 krb5kdc[22472](info): commencing operation
krb5kdc: starting...
W20260808 14:38:57.683580 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:38:57 dist-test-slave-p3t5 krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:38:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:38:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:38:57.821571 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:33107
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:35949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:33107
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:57.945930 22487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:57.946231 22487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:57.946274 22487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:57.950439 22487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:38:57.950510 22487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:57.950531 22487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:57.950548 22487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:38:57.950564 22487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:38:57.956022 22487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:35949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:33107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:33107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:57.957161 22487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:57.958334 22487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:57.968119 22492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:57.968173 22487 server_base.cc:1034] running on GCE node
W20260808 14:38:57.968125 22493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:57.968125 22495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:57.969110 22487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:57.970429 22487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:57.971637 22487 hybrid_clock.cc:648] HybridClock initialized: now 1786199937971614 us; error 36 us; skew 500 ppm
Aug 08 14:38:57 dist-test-slave-p3t5 krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199937, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:57.974805 22487 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:38:57.975800 22487 webserver.cc:460] Webserver started at http://127.19.144.126:42763/ using document root <none> and password file <none>
I20260808 14:38:57.976423 22487 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:57.976519 22487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:57.976776 22487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:57.979038 22487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "cb6bdba94b764c5db51332cef1b9683c"
format_stamp: "Formatted at 2026-08-08 14:38:57 on dist-test-slave-p3t5"
server_key: "e963bbede2edf85262390fd6ab9fb12f"
server_key_iv: "022ffb493472070e732500c8ec0029d8"
server_key_version: "encryptionkey@0"
I20260808 14:38:57.979655 22487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "cb6bdba94b764c5db51332cef1b9683c"
format_stamp: "Formatted at 2026-08-08 14:38:57 on dist-test-slave-p3t5"
server_key: "e963bbede2edf85262390fd6ab9fb12f"
server_key_iv: "022ffb493472070e732500c8ec0029d8"
server_key_version: "encryptionkey@0"
I20260808 14:38:57.983912 22487 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:38:57.986599 22502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:57.987973 22487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 14:38:57.988145 22487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "cb6bdba94b764c5db51332cef1b9683c"
format_stamp: "Formatted at 2026-08-08 14:38:57 on dist-test-slave-p3t5"
server_key: "e963bbede2edf85262390fd6ab9fb12f"
server_key_iv: "022ffb493472070e732500c8ec0029d8"
server_key_version: "encryptionkey@0"
I20260808 14:38:57.988282 22487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:58.001744 22487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:58.005441 22487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:58.005733 22487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:58.013314 22487 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:33107
I20260808 14:38:58.013338 22556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:33107 every 8 connection(s)
I20260808 14:38:58.014376 22487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:38:58.017477 22557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:38:58.019760 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 22487
I20260808 14:38:58.019901 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:38:58.020206 20033 external_mini_cluster.cc:1428] Setting key c34991c7c8c7d278481325fc81b59b05
I20260808 14:38:58.024941 22557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c: Bootstrap starting.
I20260808 14:38:58.027592 22557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c: Neither blocks nor log segments found. Creating new log.
I20260808 14:38:58.028434 22557 log.cc:824] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:38:58 dist-test-slave-p3t5 krb5kdc[22472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:58.030990 22557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c: No bootstrap required, opened a new log
I20260808 14:38:58.036077 22557 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6bdba94b764c5db51332cef1b9683c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33107 } }
I20260808 14:38:58.036386 22557 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:38:58.036450 22557 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb6bdba94b764c5db51332cef1b9683c, State: Initialized, Role: FOLLOWER
I20260808 14:38:58.036512 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:58.021896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33780 (local address 127.19.144.126:33107)
0808 14:38:58.022417 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:58.022432 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:58.022529 (+ 97us) server_negotiation.cc:407] Connection header received
0808 14:38:58.023211 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:58.023239 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:58.023632 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:58.023984 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:58.024918 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:58.026069 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:58.026986 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:58.027491 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:58.027677 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:58.030736 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:58.030743 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:58.030761 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:58.030793 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:58.032921 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:58.033462 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:58.033471 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:58.033481 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:58.033700 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:58.034042 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:58.034046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:58.034047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:58.034628 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:58.034832 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:58.035113 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 14:38:58.035458 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":158,"threads_started":1}
I20260808 14:38:58.036962 22557 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6bdba94b764c5db51332cef1b9683c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33107 } }
I20260808 14:38:58.037150 22557 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:38:58.037220 22557 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:38:58.037379 22557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:38:58.038501 22557 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6bdba94b764c5db51332cef1b9683c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33107 } }
I20260808 14:38:58.038888 22557 leader_election.cc:302] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb6bdba94b764c5db51332cef1b9683c; no voters:
I20260808 14:38:58.039255 22557 leader_election.cc:288] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:38:58.039407 22562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:38:58.039659 22562 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [term 1 LEADER]: Becoming Leader. State: Replica: cb6bdba94b764c5db51332cef1b9683c, State: Running, Role: LEADER
I20260808 14:38:58.040020 22562 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6bdba94b764c5db51332cef1b9683c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33107 } }
I20260808 14:38:58.040555 22557 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:38:58.041782 22563 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb6bdba94b764c5db51332cef1b9683c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6bdba94b764c5db51332cef1b9683c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33107 } } }
I20260808 14:38:58.041937 22563 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [sys.catalog]: This master's current role is: LEADER
I20260808 14:38:58.042325 22564 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb6bdba94b764c5db51332cef1b9683c. Latest consensus state: current_term: 1 leader_uuid: "cb6bdba94b764c5db51332cef1b9683c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6bdba94b764c5db51332cef1b9683c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33107 } } }
I20260808 14:38:58.042510 22564 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c [sys.catalog]: This master's current role is: LEADER
I20260808 14:38:58.042403 22571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:38:58.046108 22571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:38:58.052196 22571 catalog_manager.cc:1347] Generated new cluster ID: bf43c10ffbc749979a4762baae758818
I20260808 14:38:58.052289 22571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:38:58.084807 22571 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:38:58.086014 22571 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:38:58.100271 22571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb6bdba94b764c5db51332cef1b9683c: Generated new TSK 0
I20260808 14:38:58.101069 22571 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:38:58.141048 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:33107
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:35949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:58.260809 22583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:58.261116 22583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:58.261163 22583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:58.266031 22583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:58.266110 22583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:58.266173 22583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:38:58.271821 22583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:35949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:33107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:58.273037 22583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:58.274233 22583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:58.284654 22591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:58.284646 22589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:58.284646 22588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:58.284947 22583 server_base.cc:1034] running on GCE node
I20260808 14:38:58.285488 22583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:58.286221 22583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:58.287421 22583 hybrid_clock.cc:648] HybridClock initialized: now 1786199938287382 us; error 50 us; skew 500 ppm
Aug 08 14:38:58 dist-test-slave-p3t5 krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199938, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:58.290692 22583 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:38:58.291884 22583 webserver.cc:460] Webserver started at http://127.19.144.65:38213/ using document root <none> and password file <none>
I20260808 14:38:58.292573 22583 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:58.292680 22583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:58.293013 22583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:58.295547 22583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "6f3519a98f1e480db2d8abc4079d9a5f"
format_stamp: "Formatted at 2026-08-08 14:38:58 on dist-test-slave-p3t5"
server_key: "d1d1f5146705ef3b9b00d57c28ab1c91"
server_key_iv: "a63b437b8e56546e2718b4d2bbcf12c2"
server_key_version: "encryptionkey@0"
I20260808 14:38:58.296216 22583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "6f3519a98f1e480db2d8abc4079d9a5f"
format_stamp: "Formatted at 2026-08-08 14:38:58 on dist-test-slave-p3t5"
server_key: "d1d1f5146705ef3b9b00d57c28ab1c91"
server_key_iv: "a63b437b8e56546e2718b4d2bbcf12c2"
server_key_version: "encryptionkey@0"
I20260808 14:38:58.301762 22583 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 14:38:58.305177 22598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:58.306880 22583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:38:58.307011 22583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "6f3519a98f1e480db2d8abc4079d9a5f"
format_stamp: "Formatted at 2026-08-08 14:38:58 on dist-test-slave-p3t5"
server_key: "d1d1f5146705ef3b9b00d57c28ab1c91"
server_key_iv: "a63b437b8e56546e2718b4d2bbcf12c2"
server_key_version: "encryptionkey@0"
I20260808 14:38:58.307123 22583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:58.325098 22583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:58.328550 22583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:58.328729 22583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:58.329334 22583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:38:58.330314 22583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:38:58.330366 22583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:58.330410 22583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:38:58.330432 22583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:58.340035 22583 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:45057
I20260808 14:38:58.340054 22716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:45057 every 8 connection(s)
I20260808 14:38:58.341104 22583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:38:58.349781 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 22583
I20260808 14:38:58.349929 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:38:58.350216 20033 external_mini_cluster.cc:1428] Setting key fbfbdf3e4d2fc511b12aff56028136bb
Aug 08 14:38:58 dist-test-slave-p3t5 krb5kdc[22472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199938, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:58.357457 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:58.343374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:59505 (local address 127.19.144.126:33107)
0808 14:38:58.343564 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:58.343569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:58.344508 (+ 939us) server_negotiation.cc:407] Connection header received
0808 14:38:58.346023 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:58.346027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:58.346088 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:58.346177 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:58.347536 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:58.348091 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:58.349351 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:58.349604 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:58.349777 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:58.352984 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:58.352991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:58.352997 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:58.353035 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:58.354763 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:58.355521 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:58.355526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:58.355528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:58.355635 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:58.356149 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:58.356153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:58.356154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:58.356371 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:58.356465 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:58.357198 (+ 733us) server_negotiation.cc:299] Negotiation successful
0808 14:38:58.357323 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 14:38:58.358913 22719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:58.343801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:33107 (local address 127.19.144.65:59505)
0808 14:38:58.344331 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:58.344381 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:58.345782 (+ 1401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:58.346397 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:58.346404 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:58.346829 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:58.347369 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:58.347384 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:58.348238 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:58.348241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:58.349182 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:58.349192 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:58.349830 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:58.349834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:58.350088 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:58.350844 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:58.350864 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:58.352745 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:58.354909 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:58.354920 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:58.354943 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:58.355389 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:58.355744 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:58.355749 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:58.355752 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:58.355991 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:58.356495 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:58.356506 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:58.356907 (+ 401us) client_negotiation.cc:769] Sending connection context
0808 14:38:58.357157 (+ 250us) client_negotiation.cc:240] Negotiation successful
0808 14:38:58.357610 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":137,"threads_started":1}
I20260808 14:38:58.360368 22717 heartbeater.cc:347] Connected to a master server at 127.19.144.126:33107
I20260808 14:38:58.360683 22717 heartbeater.cc:464] Registering TS with master...
I20260808 14:38:58.361395 22717 heartbeater.cc:511] Master 127.19.144.126:33107 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:38:58.363346 22516 ts_manager.cc:194] Registered new tserver with Master: 6f3519a98f1e480db2d8abc4079d9a5f (127.19.144.65:45057)
I20260808 14:38:58.364786 22516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:59505
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:38:58.385221 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:33107
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:35949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:58.504290 22722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:58.504575 22722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:58.504657 22722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:58.508787 22722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:58.508858 22722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:58.508961 22722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:38:58.514592 22722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:35949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:33107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:58.515657 22722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:58.516676 22722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:58.526690 22728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:58.526690 22730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:58.526849 22727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:58.526973 22722 server_base.cc:1034] running on GCE node
I20260808 14:38:58.527468 22722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:58.528147 22722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:58.529309 22722 hybrid_clock.cc:648] HybridClock initialized: now 1786199938529275 us; error 50 us; skew 500 ppm
Aug 08 14:38:58 dist-test-slave-p3t5 krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199938, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:58.532440 22722 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:38:58.533378 22722 webserver.cc:460] Webserver started at http://127.19.144.66:35657/ using document root <none> and password file <none>
I20260808 14:38:58.534015 22722 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:58.534085 22722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:58.534261 22722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:58.536552 22722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "9990bfe19b3446eba52453ef0db323d7"
format_stamp: "Formatted at 2026-08-08 14:38:58 on dist-test-slave-p3t5"
server_key: "9af80327c89f1bd15b30d36b57c642f9"
server_key_iv: "41cdbf22f7b4b369bcc3076c3c706d06"
server_key_version: "encryptionkey@0"
I20260808 14:38:58.537099 22722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "9990bfe19b3446eba52453ef0db323d7"
format_stamp: "Formatted at 2026-08-08 14:38:58 on dist-test-slave-p3t5"
server_key: "9af80327c89f1bd15b30d36b57c642f9"
server_key_iv: "41cdbf22f7b4b369bcc3076c3c706d06"
server_key_version: "encryptionkey@0"
I20260808 14:38:58.541193 22722 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:38:58.544193 22737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:58.545531 22722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:38:58.545714 22722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "9990bfe19b3446eba52453ef0db323d7"
format_stamp: "Formatted at 2026-08-08 14:38:58 on dist-test-slave-p3t5"
server_key: "9af80327c89f1bd15b30d36b57c642f9"
server_key_iv: "41cdbf22f7b4b369bcc3076c3c706d06"
server_key_version: "encryptionkey@0"
I20260808 14:38:58.545854 22722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:58.557121 22722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:58.562146 22722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:58.562390 22722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:58.563309 22722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:38:58.564546 22722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:38:58.564611 22722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:58.564677 22722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:38:58.564708 22722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:58.574404 22722 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:38117
I20260808 14:38:58.574422 22853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:38117 every 8 connection(s)
I20260808 14:38:58.575436 22722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:38:58.583364 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 22722
I20260808 14:38:58.583519 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:38:58.583804 20033 external_mini_cluster.cc:1428] Setting key b0d2290de2b531fb711af9417dec68d3
Aug 08 14:38:58 dist-test-slave-p3t5 krb5kdc[22472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199938, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:58.591096 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:58.577467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:57575 (local address 127.19.144.126:33107)
0808 14:38:58.577660 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:58.577666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:58.578655 (+ 989us) server_negotiation.cc:407] Connection header received
0808 14:38:58.580026 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:58.580029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:58.580081 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:58.580175 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:58.581503 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:58.582181 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:58.583481 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:58.583777 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:58.583903 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:58.586956 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:58.586959 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:58.586961 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:58.586987 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:58.588732 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:58.589332 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:58.589336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:58.589338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:58.589444 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:58.589894 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:58.589898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:58.589899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:58.590102 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:58.590215 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:58.590789 (+ 574us) server_negotiation.cc:299] Negotiation successful
0808 14:38:58.590925 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:38:58.592168 22856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:58.577981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:33107 (local address 127.19.144.66:57575)
0808 14:38:58.578476 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:58.578525 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:58.579768 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:58.580346 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:58.580353 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:58.580742 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:58.581285 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:58.581301 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:58.582331 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:58.582341 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:58.583324 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:58.583335 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:58.583958 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:58.583963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:58.584146 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:58.584987 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:58.585009 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:58.586756 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:58.588875 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:58.588882 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:58.588910 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:58.589207 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:58.589549 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:58.589552 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:58.589554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:58.589782 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:58.590215 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:58.590224 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:58.590525 (+ 301us) client_negotiation.cc:769] Sending connection context
0808 14:38:58.590727 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 14:38:58.591114 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":131,"threads_started":1}
I20260808 14:38:58.593468 22854 heartbeater.cc:347] Connected to a master server at 127.19.144.126:33107
I20260808 14:38:58.593807 22854 heartbeater.cc:464] Registering TS with master...
I20260808 14:38:58.594486 22854 heartbeater.cc:511] Master 127.19.144.126:33107 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
I20260808 14:38:58.595853 22516 ts_manager.cc:194] Registered new tserver with Master: 9990bfe19b3446eba52453ef0db323d7 (127.19.144.66:38117)
I20260808 14:38:58.596520 22516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:57575
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:38:58.618078 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:33107
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:35949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:38:58.737241 22859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:38:58.737550 22859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:38:58.737597 22859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:38:58.742010 22859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:38:58.742112 22859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:38:58.742216 22859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:38:58.748327 22859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:35949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:33107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:38:58.749890 22859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:38:58.751502 22859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:38:58.762343 22867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:38:58.762347 22865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:58.762544 22859 server_base.cc:1034] running on GCE node
W20260808 14:38:58.762347 22864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:38:58.763128 22859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:38:58.763839 22859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:38:58.765040 22859 hybrid_clock.cc:648] HybridClock initialized: now 1786199938765005 us; error 53 us; skew 500 ppm
Aug 08 14:38:58 dist-test-slave-p3t5 krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199938, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:38:58.768239 22859 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:38:58.769265 22859 webserver.cc:460] Webserver started at http://127.19.144.67:34263/ using document root <none> and password file <none>
I20260808 14:38:58.769997 22859 fs_manager.cc:359] Metadata directory not provided
I20260808 14:38:58.770099 22859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:38:58.770352 22859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:38:58.772559 22859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "5af096e176a44a258960b31f888ecd20"
format_stamp: "Formatted at 2026-08-08 14:38:58 on dist-test-slave-p3t5"
server_key: "31acd2cf2defe293342d8c9b2a18e99f"
server_key_iv: "5f7e33123799b25ff73d27c457b1a090"
server_key_version: "encryptionkey@0"
I20260808 14:38:58.773203 22859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "5af096e176a44a258960b31f888ecd20"
format_stamp: "Formatted at 2026-08-08 14:38:58 on dist-test-slave-p3t5"
server_key: "31acd2cf2defe293342d8c9b2a18e99f"
server_key_iv: "5f7e33123799b25ff73d27c457b1a090"
server_key_version: "encryptionkey@0"
I20260808 14:38:58.777333 22859 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:38:58.780107 22874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:58.781487 22859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:38:58.781694 22859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "5af096e176a44a258960b31f888ecd20"
format_stamp: "Formatted at 2026-08-08 14:38:58 on dist-test-slave-p3t5"
server_key: "31acd2cf2defe293342d8c9b2a18e99f"
server_key_iv: "5f7e33123799b25ff73d27c457b1a090"
server_key_version: "encryptionkey@0"
I20260808 14:38:58.781839 22859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:38:58.803411 22859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:38:58.807291 22859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:38:58.807540 22859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:38:58.808247 22859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:38:58.809208 22859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:38:58.809288 22859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:58.809369 22859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:38:58.809414 22859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:38:58.819422 22859 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:35681
I20260808 14:38:58.819447 22990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:35681 every 8 connection(s)
I20260808 14:38:58.820433 22859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:38:58.826524 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 22859
I20260808 14:38:58.826643 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:38:58.826983 20033 external_mini_cluster.cc:1428] Setting key 1b86f8e507c5c8b91e07a6b10032c3b5
Aug 08 14:38:58 dist-test-slave-p3t5 krb5kdc[22472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199938, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:38:58.836433 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:58.822555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:58679 (local address 127.19.144.126:33107)
0808 14:38:58.822702 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:58.822705 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:58.823670 (+ 965us) server_negotiation.cc:407] Connection header received
0808 14:38:58.825120 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:58.825124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:58.825191 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:58.825273 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:58.826828 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:58.827516 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:58.829311 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:38:58.829683 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:38:58.829778 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:58.832655 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:38:58.832659 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:38:58.832661 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:38:58.832690 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:38:58.834135 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:58.834696 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:58.834699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:58.834700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:58.834789 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:38:58.835248 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:38:58.835251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:38:58.835252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:38:58.835466 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:38:58.835559 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 14:38:58.836154 (+ 595us) server_negotiation.cc:299] Negotiation successful
0808 14:38:58.836286 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:38:58.837342 22993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:38:58.822973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:33107 (local address 127.19.144.67:58679)
0808 14:38:58.823508 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 14:38:58.823555 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 14:38:58.824833 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:38:58.825467 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:38:58.825475 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:38:58.825929 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:38:58.826621 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:58.826648 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:58.827643 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:58.827649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:58.829111 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:38:58.829123 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:38:58.829847 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:38:58.829850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:38:58.830026 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:38:58.830733 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:38:58.830752 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:38:58.832459 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:38:58.834263 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:58.834269 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:58.834291 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:58.834590 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:58.834929 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:38:58.834932 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:38:58.834933 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:38:58.835105 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:38:58.835575 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:38:58.835581 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:38:58.835872 (+ 291us) client_negotiation.cc:769] Sending connection context
0808 14:38:58.836061 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 14:38:58.836372 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":127,"threads_started":1}
I20260808 14:38:58.838776 22991 heartbeater.cc:347] Connected to a master server at 127.19.144.126:33107
I20260808 14:38:58.839059 22991 heartbeater.cc:464] Registering TS with master...
I20260808 14:38:58.839671 22991 heartbeater.cc:511] Master 127.19.144.126:33107 requested a full tablet report, sending...
I20260808 14:38:58.840922 22516 ts_manager.cc:194] Registered new tserver with Master: 5af096e176a44a258960b31f888ecd20 (127.19.144.67:35681)
I20260808 14:38:58.841722 22516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:58679
I20260808 14:38:58.841882 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 14:38:58.857168 22560 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:38:58.855576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33782 (local address 127.19.144.126:33107)
0808 14:38:58.855800 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 14:38:58.855804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:38:58.855869 (+ 65us) server_negotiation.cc:407] Connection header received
0808 14:38:58.855997 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:38:58.855999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:38:58.856047 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:38:58.856125 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:38:58.856975 (+ 850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33782: BlockingRecv error: recv got EOF from 127.0.0.1:33782 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
I20260808 14:38:58.858297 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 22583
I20260808 14:38:58.867226 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 22722
I20260808 14:38:58.874574 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 22859
I20260808 14:38:58.881736 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 22487
2026-08-08T14:38:58Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2282 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:38:58 dist-test-slave-p3t5 krb5kdc[23003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:38:58 dist-test-slave-p3t5 krb5kdc[23003](info): set up 2 sockets
Aug 08 14:38:58 dist-test-slave-p3t5 krb5kdc[23003](info): commencing operation
krb5kdc: starting...
W20260808 14:38:59.947880 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:39:00 dist-test-slave-p3t5 krb5kdc[23003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:39:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:39:00.088828 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:44403
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:44403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:00.207679 23016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:00.207983 23016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:00.208027 23016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:00.212229 23016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:00.212299 23016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:00.212319 23016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:00.212337 23016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:00.212352 23016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:39:00.217967 23016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:44403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:44403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:00.219208 23016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:00.220407 23016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:39:00.230059 23016 server_base.cc:1034] running on GCE node
W20260808 14:39:00.230006 23024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:00.230015 23021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:00.229990 23022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:00.231061 23016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:00.232329 23016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:00.233554 23016 hybrid_clock.cc:648] HybridClock initialized: now 1786199940233537 us; error 29 us; skew 500 ppm
Aug 08 14:39:00 dist-test-slave-p3t5 krb5kdc[23003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199940, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:00.236711 23016 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:39:00.237732 23016 webserver.cc:460] Webserver started at http://127.19.144.126:38787/ using document root <none> and password file <none>
I20260808 14:39:00.238317 23016 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:00.238384 23016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:00.238565 23016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:00.240767 23016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "0fcc5d68ea6541a9b4b36d9c19c19b57"
format_stamp: "Formatted at 2026-08-08 14:39:00 on dist-test-slave-p3t5"
server_key: "267a88141ebd833dea5180341ac35d06"
server_key_iv: "1af2f846961c248f0eb47e81c83a12be"
server_key_version: "encryptionkey@0"
I20260808 14:39:00.241360 23016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "0fcc5d68ea6541a9b4b36d9c19c19b57"
format_stamp: "Formatted at 2026-08-08 14:39:00 on dist-test-slave-p3t5"
server_key: "267a88141ebd833dea5180341ac35d06"
server_key_iv: "1af2f846961c248f0eb47e81c83a12be"
server_key_version: "encryptionkey@0"
I20260808 14:39:00.246068 23016 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260808 14:39:00.248872 23032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:00.250447 23016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:00.250578 23016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "0fcc5d68ea6541a9b4b36d9c19c19b57"
format_stamp: "Formatted at 2026-08-08 14:39:00 on dist-test-slave-p3t5"
server_key: "267a88141ebd833dea5180341ac35d06"
server_key_iv: "1af2f846961c248f0eb47e81c83a12be"
server_key_version: "encryptionkey@0"
I20260808 14:39:00.250741 23016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:00.269115 23016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:00.272712 23016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:00.272938 23016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:00.280643 23016 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:44403
I20260808 14:39:00.280992 23086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:44403 every 8 connection(s)
I20260808 14:39:00.281759 23016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:00.285148 23087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:00.286906 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 23016
I20260808 14:39:00.287036 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:39:00.287324 20033 external_mini_cluster.cc:1428] Setting key 0c50a23e3497a917c07baa1e30e9772c
I20260808 14:39:00.292452 23087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57: Bootstrap starting.
I20260808 14:39:00.295379 23087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:00.296195 23087 log.cc:824] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:39:00 dist-test-slave-p3t5 krb5kdc[23003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:00.298782 23087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57: No bootstrap required, opened a new log
I20260808 14:39:00.304075 23087 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcc5d68ea6541a9b4b36d9c19c19b57" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 44403 } }
I20260808 14:39:00.304379 23087 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:00.304433 23087 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fcc5d68ea6541a9b4b36d9c19c19b57, State: Initialized, Role: FOLLOWER
I20260808 14:39:00.304442 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:00.288774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37964 (local address 127.19.144.126:44403)
0808 14:39:00.289237 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:00.289257 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:00.289302 (+ 45us) server_negotiation.cc:407] Connection header received
0808 14:39:00.290184 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:00.290210 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:00.290663 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:00.291063 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:00.291910 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:00.293439 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:00.294468 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:00.295186 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:00.295385 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:00.298531 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:00.298538 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:00.298557 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:00.298588 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:00.300761 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:00.301277 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:00.301284 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:00.301294 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:00.301491 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:00.301897 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:00.301901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:00.301903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:00.302489 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:00.302716 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:00.302998 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 14:39:00.303368 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":116,"threads_started":1}
I20260808 14:39:00.304962 23087 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcc5d68ea6541a9b4b36d9c19c19b57" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 44403 } }
I20260808 14:39:00.305138 23087 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:00.305233 23087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:00.305338 23087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:00.306660 23087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcc5d68ea6541a9b4b36d9c19c19b57" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 44403 } }
I20260808 14:39:00.307085 23087 leader_election.cc:302] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fcc5d68ea6541a9b4b36d9c19c19b57; no voters:
I20260808 14:39:00.307472 23087 leader_election.cc:288] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:39:00.307546 23092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:00.308038 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:39:00.307858 23092 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [term 1 LEADER]: Becoming Leader. State: Replica: 0fcc5d68ea6541a9b4b36d9c19c19b57, State: Running, Role: LEADER
I20260808 14:39:00.308451 23092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcc5d68ea6541a9b4b36d9c19c19b57" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 44403 } }
I20260808 14:39:00.308971 23087 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:00.309901 23092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fcc5d68ea6541a9b4b36d9c19c19b57. Latest consensus state: current_term: 1 leader_uuid: "0fcc5d68ea6541a9b4b36d9c19c19b57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcc5d68ea6541a9b4b36d9c19c19b57" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 44403 } } }
I20260808 14:39:00.310026 23092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:00.311789 23101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:00.313077 23093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fcc5d68ea6541a9b4b36d9c19c19b57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcc5d68ea6541a9b4b36d9c19c19b57" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 44403 } } }
I20260808 14:39:00.313306 23093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:00.315613 23101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:00.322671 23101 catalog_manager.cc:1347] Generated new cluster ID: 2528919be8c34e09a587d18b634c780d
I20260808 14:39:00.322750 23101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:00.333562 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:00.327529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37970 (local address 127.19.144.126:44403)
0808 14:39:00.327638 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:00.327643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:00.327935 (+ 292us) server_negotiation.cc:407] Connection header received
0808 14:39:00.328052 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:00.328056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:00.328117 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:00.328213 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:00.328847 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:00.329719 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:00.330878 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:00.331248 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:00.331356 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:00.332383 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:00.332388 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:00.332391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:00.333032 (+ 641us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:00.333071 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:00.333094 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:00.333189 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:00.333330 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 14:39:00.333397 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":37}
I20260808 14:39:00.334010 23101 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:39:00.335274 23101 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:00.344620 23101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fcc5d68ea6541a9b4b36d9c19c19b57: Generated new TSK 0
I20260808 14:39:00.345402 23101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:39:00.357441 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 23016
2026-08-08T14:39:00Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1473 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:39:00 dist-test-slave-p3t5 krb5kdc[23121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:39:00 dist-test-slave-p3t5 krb5kdc[23121](info): set up 2 sockets
Aug 08 14:39:00 dist-test-slave-p3t5 krb5kdc[23121](info): commencing operation
krb5kdc: starting...
W20260808 14:39:01.458945 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:39:01 dist-test-slave-p3t5 krb5kdc[23121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:39:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:39:01.602074 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:46371
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:46371
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:01.720794 23136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:01.721120 23136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:01.721166 23136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:01.725306 23136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:01.725374 23136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:01.725395 23136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:01.725412 23136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:01.725427 23136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:39:01.731105 23136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:46371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:46371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:01.732236 23136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:01.733285 23136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:01.742458 23142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:01.742458 23145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:01.742504 23143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:01.742719 23136 server_base.cc:1034] running on GCE node
I20260808 14:39:01.743341 23136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:01.744513 23136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:01.745708 23136 hybrid_clock.cc:648] HybridClock initialized: now 1786199941745671 us; error 49 us; skew 500 ppm
Aug 08 14:39:01 dist-test-slave-p3t5 krb5kdc[23121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199941, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:01.753453 23136 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:39:01.754665 23136 webserver.cc:460] Webserver started at http://127.19.144.126:45989/ using document root <none> and password file <none>
I20260808 14:39:01.755311 23136 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:01.755411 23136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:01.755672 23136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:01.757998 23136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "f23dc4648d30420fae772d4b159b9e9d"
format_stamp: "Formatted at 2026-08-08 14:39:01 on dist-test-slave-p3t5"
server_key: "374a91c8183a91f6925fac892029fbd4"
server_key_iv: "f32029e2951f87a0ca805a9a1796a415"
server_key_version: "encryptionkey@0"
I20260808 14:39:01.758641 23136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "f23dc4648d30420fae772d4b159b9e9d"
format_stamp: "Formatted at 2026-08-08 14:39:01 on dist-test-slave-p3t5"
server_key: "374a91c8183a91f6925fac892029fbd4"
server_key_iv: "f32029e2951f87a0ca805a9a1796a415"
server_key_version: "encryptionkey@0"
I20260808 14:39:01.762763 23136 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:39:01.765461 23152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:01.766885 23136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:39:01.767053 23136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "f23dc4648d30420fae772d4b159b9e9d"
format_stamp: "Formatted at 2026-08-08 14:39:01 on dist-test-slave-p3t5"
server_key: "374a91c8183a91f6925fac892029fbd4"
server_key_iv: "f32029e2951f87a0ca805a9a1796a415"
server_key_version: "encryptionkey@0"
I20260808 14:39:01.767187 23136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:01.781270 23136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:01.784768 23136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:01.785022 23136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:01.792940 23136 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:46371
I20260808 14:39:01.793365 23204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:46371 every 8 connection(s)
I20260808 14:39:01.794133 23136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:01.797252 23205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:01.800452 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 23136
I20260808 14:39:01.800585 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:39:01.800882 20033 external_mini_cluster.cc:1428] Setting key 1d60bbe23210bbdcb87586a30a03d1fe
I20260808 14:39:01.803920 23205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d: Bootstrap starting.
I20260808 14:39:01.808396 23205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:01.809415 23205 log.cc:824] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:39:01 dist-test-slave-p3t5 krb5kdc[23121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:01.811733 23205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d: No bootstrap required, opened a new log
I20260808 14:39:01.815644 23205 raft_consensus.cc:348] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f23dc4648d30420fae772d4b159b9e9d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46371 } }
I20260808 14:39:01.815961 23205 raft_consensus.cc:374] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:01.816044 23205 raft_consensus.cc:729] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f23dc4648d30420fae772d4b159b9e9d, State: Initialized, Role: FOLLOWER
I20260808 14:39:01.816511 23205 consensus_queue.cc:260] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f23dc4648d30420fae772d4b159b9e9d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46371 } }
I20260808 14:39:01.816669 23205 raft_consensus.cc:388] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:01.816767 23205 raft_consensus.cc:482] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:01.816802 23208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:01.802366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48006 (local address 127.19.144.126:46371)
0808 14:39:01.802857 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:01.802887 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:01.802924 (+ 37us) server_negotiation.cc:407] Connection header received
0808 14:39:01.803540 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:01.803558 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:01.803923 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:01.804260 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:01.805168 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:01.806910 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:01.808091 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:01.808656 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:01.808798 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:01.811690 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:01.811696 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:01.811710 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:01.811739 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:01.813533 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:01.814067 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:01.814073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:01.814078 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:01.814196 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:01.814504 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:01.814508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:01.814510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:01.814934 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:01.815170 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:01.815440 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 14:39:01.815754 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":182,"threads_started":1}
I20260808 14:39:01.816893 23205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:01.818150 23205 raft_consensus.cc:504] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f23dc4648d30420fae772d4b159b9e9d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46371 } }
I20260808 14:39:01.818514 23205 leader_election.cc:302] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f23dc4648d30420fae772d4b159b9e9d; no voters:
I20260808 14:39:01.818843 23205 leader_election.cc:288] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:39:01.818956 23210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:01.819192 23210 raft_consensus.cc:686] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [term 1 LEADER]: Becoming Leader. State: Replica: f23dc4648d30420fae772d4b159b9e9d, State: Running, Role: LEADER
I20260808 14:39:01.819607 23210 consensus_queue.cc:237] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f23dc4648d30420fae772d4b159b9e9d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46371 } }
I20260808 14:39:01.820185 23205 sys_catalog.cc:564] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:01.820834 23210 sys_catalog.cc:455] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f23dc4648d30420fae772d4b159b9e9d. Latest consensus state: current_term: 1 leader_uuid: "f23dc4648d30420fae772d4b159b9e9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f23dc4648d30420fae772d4b159b9e9d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46371 } } }
I20260808 14:39:01.821024 23210 sys_catalog.cc:458] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:01.821262 23211 sys_catalog.cc:455] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f23dc4648d30420fae772d4b159b9e9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f23dc4648d30420fae772d4b159b9e9d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46371 } } }
I20260808 14:39:01.821375 23211 sys_catalog.cc:458] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:01.821904 23219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:01.824738 23219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:01.830536 23219 catalog_manager.cc:1347] Generated new cluster ID: 2e80c8ae56de4330bf8162b181afaad0
I20260808 14:39:01.830612 23219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:01.849908 23219 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:39:01.850971 23219 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:01.855305 23219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f23dc4648d30420fae772d4b159b9e9d: Generated new TSK 0
I20260808 14:39:01.856186 23219 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:39:01.893721 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46371
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:02.013685 23233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:02.014052 23233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:02.014134 23233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:02.018209 23233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:02.018319 23233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:02.018414 23233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:39:02.024199 23233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:02.025467 23233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:02.026665 23233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:02.037242 23238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:02.037304 23233 server_base.cc:1034] running on GCE node
W20260808 14:39:02.037235 23239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:02.037559 23242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:02.038026 23233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:02.038753 23233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:02.039975 23233 hybrid_clock.cc:648] HybridClock initialized: now 1786199942039954 us; error 32 us; skew 500 ppm
Aug 08 14:39:02 dist-test-slave-p3t5 krb5kdc[23121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199942, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:02.043169 23233 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:39:02.044092 23233 webserver.cc:460] Webserver started at http://127.19.144.65:46289/ using document root <none> and password file <none>
I20260808 14:39:02.044663 23233 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:02.044723 23233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:02.044904 23233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:02.047104 23233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "69c7c380735f448cae404fadc9d86ac2"
format_stamp: "Formatted at 2026-08-08 14:39:02 on dist-test-slave-p3t5"
server_key: "3d9eaf65def75ae0ea626ebd0740a277"
server_key_iv: "5e46f28e07c407cc08f32132762aa2f3"
server_key_version: "encryptionkey@0"
I20260808 14:39:02.047676 23233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "69c7c380735f448cae404fadc9d86ac2"
format_stamp: "Formatted at 2026-08-08 14:39:02 on dist-test-slave-p3t5"
server_key: "3d9eaf65def75ae0ea626ebd0740a277"
server_key_iv: "5e46f28e07c407cc08f32132762aa2f3"
server_key_version: "encryptionkey@0"
I20260808 14:39:02.051690 23233 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:39:02.054303 23249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:02.055660 23233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:02.055815 23233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "69c7c380735f448cae404fadc9d86ac2"
format_stamp: "Formatted at 2026-08-08 14:39:02 on dist-test-slave-p3t5"
server_key: "3d9eaf65def75ae0ea626ebd0740a277"
server_key_iv: "5e46f28e07c407cc08f32132762aa2f3"
server_key_version: "encryptionkey@0"
I20260808 14:39:02.055945 23233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:02.067870 23233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:02.071717 23233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:02.071949 23233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:02.072633 23233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:02.073563 23233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:02.073664 23233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:02.073766 23233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:02.073819 23233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:02.083803 23233 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:38699
I20260808 14:39:02.083856 23364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:38699 every 8 connection(s)
I20260808 14:39:02.084807 23233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:39:02.092209 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 23233
I20260808 14:39:02.092368 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:39:02.092684 20033 external_mini_cluster.cc:1428] Setting key 17b4854ff4dd70cac04844972d6a885d
Aug 08 14:39:02 dist-test-slave-p3t5 krb5kdc[23121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199942, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:02.101068 23208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:02.086854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:51785 (local address 127.19.144.126:46371)
0808 14:39:02.087007 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:02.087011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:02.087825 (+ 814us) server_negotiation.cc:407] Connection header received
0808 14:39:02.089342 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:02.089345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:02.089410 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:02.089518 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:02.090900 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:02.091474 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:02.093118 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:02.093448 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:02.093583 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:02.096582 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:02.096585 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:02.096587 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:02.096613 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:02.098552 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:02.099150 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:02.099157 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:02.099161 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:02.099270 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:02.099662 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:02.099668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:02.099673 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:02.099960 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:02.100082 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:02.100631 (+ 549us) server_negotiation.cc:299] Negotiation successful
0808 14:39:02.100798 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:39:02.101922 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:02.087184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46371 (local address 127.19.144.65:51785)
0808 14:39:02.087667 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:02.087712 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:02.089108 (+ 1396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:02.089739 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:02.089746 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:02.090147 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:02.090736 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:02.090753 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:02.091620 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:02.091623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:02.092964 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:02.092976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:02.093662 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:02.093668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:02.093859 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:02.094554 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:02.094571 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:02.096351 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:02.098684 (+ 2333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:02.098690 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:02.098705 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:02.099035 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:02.099382 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:02.099384 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:02.099385 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:02.099537 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:02.100082 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:02.100088 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:02.100366 (+ 278us) client_negotiation.cc:769] Sending connection context
0808 14:39:02.100540 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 14:39:02.100838 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":134,"threads_started":1}
I20260808 14:39:02.103287 23365 heartbeater.cc:347] Connected to a master server at 127.19.144.126:46371
I20260808 14:39:02.103549 23365 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:02.104086 23365 heartbeater.cc:511] Master 127.19.144.126:46371 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:39:02.105901 23169 ts_manager.cc:194] Registered new tserver with Master: 69c7c380735f448cae404fadc9d86ac2 (127.19.144.65:38699)
I20260808 14:39:02.107304 23169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:51785
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:39:02.124889 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46371
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:02.244998 23370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:02.245481 23370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:02.245587 23370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:02.250061 23370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:02.250141 23370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:02.250204 23370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:39:02.255815 23370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:02.256976 23370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:02.258160 23370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:02.268792 23375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:02.269002 23370 server_base.cc:1034] running on GCE node
W20260808 14:39:02.268792 23376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:02.268814 23378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:02.269605 23370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:02.270469 23370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:02.271644 23370 hybrid_clock.cc:648] HybridClock initialized: now 1786199942271609 us; error 50 us; skew 500 ppm
Aug 08 14:39:02 dist-test-slave-p3t5 krb5kdc[23121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199942, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:02.274672 23370 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:39:02.275667 23370 webserver.cc:460] Webserver started at http://127.19.144.66:37327/ using document root <none> and password file <none>
I20260808 14:39:02.276221 23370 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:02.276285 23370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:02.276469 23370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:02.278759 23370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "724b63303f444c2ba3aad54e0c5071cf"
format_stamp: "Formatted at 2026-08-08 14:39:02 on dist-test-slave-p3t5"
server_key: "0f746ad7b3a1c463a4f03b9b571a2295"
server_key_iv: "90f59cc21c891948362f78eb975b6a44"
server_key_version: "encryptionkey@0"
I20260808 14:39:02.279345 23370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "724b63303f444c2ba3aad54e0c5071cf"
format_stamp: "Formatted at 2026-08-08 14:39:02 on dist-test-slave-p3t5"
server_key: "0f746ad7b3a1c463a4f03b9b571a2295"
server_key_iv: "90f59cc21c891948362f78eb975b6a44"
server_key_version: "encryptionkey@0"
I20260808 14:39:02.283597 23370 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 14:39:02.286294 23385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:02.287765 23370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:02.287918 23370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "724b63303f444c2ba3aad54e0c5071cf"
format_stamp: "Formatted at 2026-08-08 14:39:02 on dist-test-slave-p3t5"
server_key: "0f746ad7b3a1c463a4f03b9b571a2295"
server_key_iv: "90f59cc21c891948362f78eb975b6a44"
server_key_version: "encryptionkey@0"
I20260808 14:39:02.288058 23370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:02.303272 23370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:02.306843 23370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:02.307029 23370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:02.307621 23370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:02.308468 23370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:02.308517 23370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:02.308560 23370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:02.308576 23370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:02.318320 23370 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:41105
I20260808 14:39:02.318382 23501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:41105 every 8 connection(s)
I20260808 14:39:02.319376 23370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:39:02.323284 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 23370
I20260808 14:39:02.323419 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:39:02.324429 20033 external_mini_cluster.cc:1428] Setting key 255e40fd998bee498eda11b17d3008bf
Aug 08 14:39:02 dist-test-slave-p3t5 krb5kdc[23121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199942, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
I20260808 14:39:02.337181 23208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:02.321769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:45631 (local address 127.19.144.126:46371)
0808 14:39:02.321924 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:02.321928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:02.323184 (+ 1256us) server_negotiation.cc:407] Connection header received
0808 14:39:02.325276 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:02.325280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:02.325359 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:02.325424 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:02.327719 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:02.328305 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:02.329776 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:02.330034 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:02.330156 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:02.333082 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:02.333086 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:02.333088 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:02.333117 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:02.334728 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:02.335332 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:02.335335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:02.335337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:02.335422 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:02.335858 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:02.335866 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:02.335867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:02.336070 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:02.336183 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:02.336831 (+ 648us) server_negotiation.cc:299] Negotiation successful
0808 14:39:02.336994 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:39:02.338299 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:02.322276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46371 (local address 127.19.144.66:45631)
0808 14:39:02.322984 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:02.323054 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:02.324969 (+ 1915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:02.325973 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:02.325983 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:02.326604 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:02.327510 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:02.327539 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:02.328458 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:02.328465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:02.329599 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:02.329610 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:02.330221 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:02.330225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:02.330434 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:02.331135 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:02.331153 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:02.332851 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:02.334869 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:02.334878 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:02.334899 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:02.335200 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:02.335528 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:02.335531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:02.335533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:02.335737 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:02.336184 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:02.336193 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:02.336535 (+ 342us) client_negotiation.cc:769] Sending connection context
0808 14:39:02.336770 (+ 235us) client_negotiation.cc:240] Negotiation successful
0808 14:39:02.337154 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":166,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
I20260808 14:39:02.339872 23502 heartbeater.cc:347] Connected to a master server at 127.19.144.126:46371
I20260808 14:39:02.340171 23502 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:02.340859 23502 heartbeater.cc:511] Master 127.19.144.126:46371 requested a full tablet report, sending...
I20260808 14:39:02.342247 23169 ts_manager.cc:194] Registered new tserver with Master: 724b63303f444c2ba3aad54e0c5071cf (127.19.144.66:41105)
I20260808 14:39:02.343183 23169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:45631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:39:02.358101 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46371
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:02.477175 23508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:02.477520 23508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:02.477600 23508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:02.481974 23508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:02.482075 23508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:02.482173 23508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:39:02.487777 23508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:02.488996 23508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:02.490197 23508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:39:02.500916 23508 server_base.cc:1034] running on GCE node
W20260808 14:39:02.500890 23514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:02.500859 23513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:02.500859 23516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:02.501756 23508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:02.502466 23508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:02.503697 23508 hybrid_clock.cc:648] HybridClock initialized: now 1786199942503679 us; error 27 us; skew 500 ppm
Aug 08 14:39:02 dist-test-slave-p3t5 krb5kdc[23121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199942, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:02.507102 23508 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:39:02.508028 23508 webserver.cc:460] Webserver started at http://127.19.144.67:40773/ using document root <none> and password file <none>
I20260808 14:39:02.508607 23508 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:02.508673 23508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:02.508872 23508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:02.511148 23508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "f5e3c279934d4975bd7e81844c3d7144"
format_stamp: "Formatted at 2026-08-08 14:39:02 on dist-test-slave-p3t5"
server_key: "f71e472084e68df6cede2bc7eb450115"
server_key_iv: "b53c13c09b000fdee41cb51eb92e42d6"
server_key_version: "encryptionkey@0"
I20260808 14:39:02.511719 23508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "f5e3c279934d4975bd7e81844c3d7144"
format_stamp: "Formatted at 2026-08-08 14:39:02 on dist-test-slave-p3t5"
server_key: "f71e472084e68df6cede2bc7eb450115"
server_key_iv: "b53c13c09b000fdee41cb51eb92e42d6"
server_key_version: "encryptionkey@0"
I20260808 14:39:02.515841 23508 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 14:39:02.518431 23523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:02.519826 23508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:39:02.519936 23508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "f5e3c279934d4975bd7e81844c3d7144"
format_stamp: "Formatted at 2026-08-08 14:39:02 on dist-test-slave-p3t5"
server_key: "f71e472084e68df6cede2bc7eb450115"
server_key_iv: "b53c13c09b000fdee41cb51eb92e42d6"
server_key_version: "encryptionkey@0"
I20260808 14:39:02.520038 23508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:02.541424 23508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:02.545998 23508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:02.546226 23508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:02.546876 23508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:02.547787 23508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:02.547861 23508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:02.547938 23508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:02.547977 23508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:02.560312 23508 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:40059
I20260808 14:39:02.560367 23639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:40059 every 8 connection(s)
I20260808 14:39:02.561789 23508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:39:02.566607 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 23508
I20260808 14:39:02.566751 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:39:02.567060 20033 external_mini_cluster.cc:1428] Setting key dd346d0aaecca7dce4f401edc16f2b3f
Aug 08 14:39:02 dist-test-slave-p3t5 krb5kdc[23121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199942, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:02.584183 23208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:02.570040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:38161 (local address 127.19.144.126:46371)
0808 14:39:02.570174 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:02.570180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:02.571513 (+ 1333us) server_negotiation.cc:407] Connection header received
0808 14:39:02.573790 (+ 2277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:02.573794 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:02.573876 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:02.574000 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:02.575477 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:02.576017 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:02.577186 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:02.577407 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:02.577487 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:02.580118 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:02.580121 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:02.580123 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:02.580151 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:02.581992 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:02.582547 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:02.582551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:02.582553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:02.582649 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:02.583027 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:02.583030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:02.583032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:02.583235 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:02.583355 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:02.583882 (+ 527us) server_negotiation.cc:299] Negotiation successful
0808 14:39:02.584016 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 14:39:02.585109 23642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:02.570614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46371 (local address 127.19.144.67:38161)
0808 14:39:02.571322 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:02.571404 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:02.573493 (+ 2089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:02.574250 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:02.574260 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:02.574801 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:02.575332 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:02.575349 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:02.576134 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:02.576136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:02.577077 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:02.577086 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:02.577544 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:02.577547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:02.577779 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:02.578376 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:02.578393 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:02.579953 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:02.582125 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:02.582132 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:02.582148 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:02.582432 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:02.582748 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:02.582752 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:02.582754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:02.582921 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:02.583357 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:02.583366 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:02.583634 (+ 268us) client_negotiation.cc:769] Sending connection context
0808 14:39:02.583836 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 14:39:02.584205 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":168,"threads_started":1}
I20260808 14:39:02.586333 23640 heartbeater.cc:347] Connected to a master server at 127.19.144.126:46371
I20260808 14:39:02.586609 23640 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:02.587173 23640 heartbeater.cc:511] Master 127.19.144.126:46371 requested a full tablet report, sending...
I20260808 14:39:02.588271 23169 ts_manager.cc:194] Registered new tserver with Master: f5e3c279934d4975bd7e81844c3d7144 (127.19.144.67:40059)
I20260808 14:39:02.588882 23169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:38161
I20260808 14:39:02.594761 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 14:39:02 dist-test-slave-p3t5 krb5kdc[23121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199942, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 14:39:02 dist-test-slave-p3t5 krb5kdc[23121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199942, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:02.623605 23208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:02.614522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48022 (local address 127.19.144.126:46371)
0808 14:39:02.614728 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:02.614732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:02.614800 (+ 68us) server_negotiation.cc:407] Connection header received
0808 14:39:02.614942 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:02.614944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:02.614992 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:02.615066 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:02.615819 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:02.616364 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:02.617167 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:02.617381 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:02.617456 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:02.619966 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:02.619969 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:02.619971 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:02.620000 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:02.621713 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:02.622156 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:02.622159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:02.622161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:02.622242 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:02.622612 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:02.622614 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:02.622615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:02.622857 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:02.622988 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:02.623266 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 14:39:02.623405 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260808 14:39:02.624066 23169 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48022
I20260808 14:39:02.625005 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 23233
I20260808 14:39:02.635713 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 23370
I20260808 14:39:02.642972 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 23508
I20260808 14:39:02.650668 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 23136
2026-08-08T14:39:02Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2294 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:39:02 dist-test-slave-p3t5 krb5kdc[23653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:39:02 dist-test-slave-p3t5 krb5kdc[23653](info): set up 2 sockets
Aug 08 14:39:02 dist-test-slave-p3t5 krb5kdc[23653](info): commencing operation
krb5kdc: starting...
W20260808 14:39:03.719550 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:39:03 dist-test-slave-p3t5 krb5kdc[23653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:39:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:39:03.858871 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:46287
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:43107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:46287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:03.975843 23666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:03.976220 23666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:03.976320 23666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:03.980520 23666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:03.980640 23666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:03.980682 23666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:03.980720 23666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:03.980757 23666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:39:03.986817 23666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:43107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:46287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:46287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:03.988063 23666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:03.989207 23666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:03.998327 23671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:03.998327 23672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:03.998466 23666 server_base.cc:1034] running on GCE node
W20260808 14:39:03.998319 23674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:03.999226 23666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:04.000396 23666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:04.001595 23666 hybrid_clock.cc:648] HybridClock initialized: now 1786199944001564 us; error 44 us; skew 500 ppm
Aug 08 14:39:04 dist-test-slave-p3t5 krb5kdc[23653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199944, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:04.004619 23666 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:39:04.005573 23666 webserver.cc:460] Webserver started at http://127.19.144.126:34051/ using document root <none> and password file <none>
I20260808 14:39:04.006260 23666 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:04.006356 23666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:04.006614 23666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:04.008838 23666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "e72007c5ab02416099cb37f15ea698b8"
format_stamp: "Formatted at 2026-08-08 14:39:04 on dist-test-slave-p3t5"
server_key: "1abd92682146d98cdfaa8b80b4443f26"
server_key_iv: "90fa19fff4624ab84f200e2d9ab45dff"
server_key_version: "encryptionkey@0"
I20260808 14:39:04.009483 23666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "e72007c5ab02416099cb37f15ea698b8"
format_stamp: "Formatted at 2026-08-08 14:39:04 on dist-test-slave-p3t5"
server_key: "1abd92682146d98cdfaa8b80b4443f26"
server_key_iv: "90fa19fff4624ab84f200e2d9ab45dff"
server_key_version: "encryptionkey@0"
I20260808 14:39:04.013490 23666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260808 14:39:04.016206 23681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:04.017557 23666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:39:04.017753 23666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "e72007c5ab02416099cb37f15ea698b8"
format_stamp: "Formatted at 2026-08-08 14:39:04 on dist-test-slave-p3t5"
server_key: "1abd92682146d98cdfaa8b80b4443f26"
server_key_iv: "90fa19fff4624ab84f200e2d9ab45dff"
server_key_version: "encryptionkey@0"
I20260808 14:39:04.017861 23666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:04.037393 23666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:04.041044 23666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:04.041219 23666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:04.048828 23666 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:46287
I20260808 14:39:04.048848 23734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:46287 every 8 connection(s)
I20260808 14:39:04.049957 23666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:04.052955 23735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:04.057158 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 23666
I20260808 14:39:04.057268 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:39:04.057559 20033 external_mini_cluster.cc:1428] Setting key 3097b8420b6cf3a6f580a1aa9e6e150c
I20260808 14:39:04.059154 23735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8: Bootstrap starting.
I20260808 14:39:04.061492 23735 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:04.062294 23735 log.cc:824] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:04.064328 23735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8: No bootstrap required, opened a new log
Aug 08 14:39:04 dist-test-slave-p3t5 krb5kdc[23653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:04.068562 23735 raft_consensus.cc:348] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72007c5ab02416099cb37f15ea698b8" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46287 } }
I20260808 14:39:04.068835 23735 raft_consensus.cc:374] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:04.068929 23735 raft_consensus.cc:729] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72007c5ab02416099cb37f15ea698b8, State: Initialized, Role: FOLLOWER
I20260808 14:39:04.069403 23735 consensus_queue.cc:260] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72007c5ab02416099cb37f15ea698b8" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46287 } }
I20260808 14:39:04.069581 23735 raft_consensus.cc:388] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:04.069716 23735 raft_consensus.cc:482] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:04.069854 23735 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:04.071151 23735 raft_consensus.cc:504] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72007c5ab02416099cb37f15ea698b8" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46287 } }
I20260808 14:39:04.071516 23735 leader_election.cc:302] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e72007c5ab02416099cb37f15ea698b8; no voters:
I20260808 14:39:04.071909 23735 leader_election.cc:288] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:39:04.072079 23740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:04.072337 23740 raft_consensus.cc:686] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [term 1 LEADER]: Becoming Leader. State: Replica: e72007c5ab02416099cb37f15ea698b8, State: Running, Role: LEADER
I20260808 14:39:04.072739 23740 consensus_queue.cc:237] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72007c5ab02416099cb37f15ea698b8" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46287 } }
I20260808 14:39:04.073227 23735 sys_catalog.cc:564] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:04.073459 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:04.059063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48316 (local address 127.19.144.126:46287)
0808 14:39:04.059528 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:04.059543 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:04.059581 (+ 38us) server_negotiation.cc:407] Connection header received
0808 14:39:04.060241 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:04.060260 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:04.060609 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:04.060967 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:04.061902 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.063180 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.064074 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.064624 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.064809 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:04.067866 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:04.067871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:04.067910 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:04.067937 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:04.070043 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:04.070560 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:04.070565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:04.070570 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:04.070691 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:04.071019 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:04.071022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:04.071023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:04.071498 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:04.071676 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:04.072071 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 14:39:04.072414 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":154,"threads_started":1}
I20260808 14:39:04.074445 23742 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e72007c5ab02416099cb37f15ea698b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72007c5ab02416099cb37f15ea698b8" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46287 } } }
I20260808 14:39:04.074574 23742 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:04.074805 23742 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e72007c5ab02416099cb37f15ea698b8. Latest consensus state: current_term: 1 leader_uuid: "e72007c5ab02416099cb37f15ea698b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72007c5ab02416099cb37f15ea698b8" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46287 } } }
I20260808 14:39:04.074890 23742 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:04.075881 23749 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:04.078835 23749 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:04.084826 23749 catalog_manager.cc:1347] Generated new cluster ID: 96e4ab03dd43446592f769a4375afb95
I20260808 14:39:04.084901 23749 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:04.102221 23749 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:39:04.103637 23749 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:04.108417 23749 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e72007c5ab02416099cb37f15ea698b8: Generated new TSK 0
I20260808 14:39:04.109144 23749 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:39:04.142867 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46287
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:43107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:04.263152 23762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:04.263500 23762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:04.263581 23762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:04.267835 23762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:04.267941 23762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:04.268040 23762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:39:04.273667 23762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:43107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:04.275017 23762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:04.276175 23762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:39:04.286639 23762 server_base.cc:1034] running on GCE node
W20260808 14:39:04.286504 23770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:04.286576 23768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:04.286504 23767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:04.287276 23762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:04.288028 23762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:04.289247 23762 hybrid_clock.cc:648] HybridClock initialized: now 1786199944289230 us; error 29 us; skew 500 ppm
Aug 08 14:39:04 dist-test-slave-p3t5 krb5kdc[23653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199944, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:04.292891 23762 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:39:04.293998 23762 webserver.cc:460] Webserver started at http://127.19.144.65:34075/ using document root <none> and password file <none>
I20260808 14:39:04.294641 23762 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:04.294744 23762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:04.295009 23762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:04.297406 23762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "64d006162df342df83788840ce3ab649"
format_stamp: "Formatted at 2026-08-08 14:39:04 on dist-test-slave-p3t5"
server_key: "bcda9bf21cc93c96d51087dbae0807d5"
server_key_iv: "ea2df79608c63396baff9d4bb13ad242"
server_key_version: "encryptionkey@0"
I20260808 14:39:04.298115 23762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "64d006162df342df83788840ce3ab649"
format_stamp: "Formatted at 2026-08-08 14:39:04 on dist-test-slave-p3t5"
server_key: "bcda9bf21cc93c96d51087dbae0807d5"
server_key_iv: "ea2df79608c63396baff9d4bb13ad242"
server_key_version: "encryptionkey@0"
I20260808 14:39:04.302671 23762 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:39:04.305493 23777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:04.307046 23762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:04.307227 23762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "64d006162df342df83788840ce3ab649"
format_stamp: "Formatted at 2026-08-08 14:39:04 on dist-test-slave-p3t5"
server_key: "bcda9bf21cc93c96d51087dbae0807d5"
server_key_iv: "ea2df79608c63396baff9d4bb13ad242"
server_key_version: "encryptionkey@0"
I20260808 14:39:04.307379 23762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:04.325007 23762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:04.328397 23762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:04.328634 23762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:04.329293 23762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:04.330276 23762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:04.330355 23762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:04.330438 23762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:04.330490 23762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:04.341094 23762 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:46263
I20260808 14:39:04.341112 23896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:46263 every 8 connection(s)
I20260808 14:39:04.342183 23762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:39:04.351295 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 23762
I20260808 14:39:04.351414 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:39:04.351727 20033 external_mini_cluster.cc:1428] Setting key 96f0b1d836e316bcff3aadf184222dff
Aug 08 14:39:04 dist-test-slave-p3t5 krb5kdc[23653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199944, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:04.357419 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:04.344270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:43903 (local address 127.19.144.126:46287)
0808 14:39:04.344429 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:04.344433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:04.345322 (+ 889us) server_negotiation.cc:407] Connection header received
0808 14:39:04.346896 (+ 1574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:04.346898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:04.346949 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:04.347027 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:04.348279 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.348810 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.350051 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.350281 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.350378 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:04.353082 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:04.353091 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:04.353093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:04.353117 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:04.354888 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:04.355420 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:04.355426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:04.355431 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:04.355543 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:04.356037 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:04.356040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:04.356041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:04.356229 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:04.356309 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:04.357011 (+ 702us) server_negotiation.cc:299] Negotiation successful
0808 14:39:04.357172 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 14:39:04.358326 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:04.344672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46287 (local address 127.19.144.65:43903)
0808 14:39:04.345159 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:04.345205 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:04.346659 (+ 1454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:04.347219 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:04.347226 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:04.347641 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:04.348140 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:04.348154 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.348928 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.348931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:04.349938 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:04.349947 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.350436 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.350439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:04.350590 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:04.351265 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:04.351282 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:04.352903 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:04.355014 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:04.355020 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:04.355040 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:04.355313 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:04.355670 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:04.355676 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:04.355678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:04.355931 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:04.356339 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:04.356347 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:04.356715 (+ 368us) client_negotiation.cc:769] Sending connection context
0808 14:39:04.356973 (+ 258us) client_negotiation.cc:240] Negotiation successful
0808 14:39:04.357354 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":129,"threads_started":1}
I20260808 14:39:04.359722 23897 heartbeater.cc:347] Connected to a master server at 127.19.144.126:46287
I20260808 14:39:04.359982 23897 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:04.360518 23897 heartbeater.cc:511] Master 127.19.144.126:46287 requested a full tablet report, sending...
I20260808 14:39:04.362156 23695 ts_manager.cc:194] Registered new tserver with Master: 64d006162df342df83788840ce3ab649 (127.19.144.65:46263)
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:39:04.363417 23695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:43903
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:39:04.384631 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46287
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:43107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:04.503396 23902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:04.503691 23902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:04.503738 23902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:04.507810 23902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:04.507887 23902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:04.507948 23902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:39:04.513583 23902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:43107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:04.514708 23902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:04.515836 23902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:04.526247 23908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:04.526263 23910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:04.526376 23902 server_base.cc:1034] running on GCE node
W20260808 14:39:04.526273 23907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:04.526986 23902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:04.527716 23902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:04.528925 23902 hybrid_clock.cc:648] HybridClock initialized: now 1786199944528911 us; error 34 us; skew 500 ppm
Aug 08 14:39:04 dist-test-slave-p3t5 krb5kdc[23653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199944, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:04.531909 23902 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:39:04.532912 23902 webserver.cc:460] Webserver started at http://127.19.144.66:41219/ using document root <none> and password file <none>
I20260808 14:39:04.533517 23902 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:04.533615 23902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:04.533896 23902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:04.536059 23902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "eb0b711778c147818c98d21e9841db45"
format_stamp: "Formatted at 2026-08-08 14:39:04 on dist-test-slave-p3t5"
server_key: "1a505d695cc6f8a5eb31dc0e81acb496"
server_key_iv: "c5c8ed44900d85e96b891061381f80cd"
server_key_version: "encryptionkey@0"
I20260808 14:39:04.536674 23902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "eb0b711778c147818c98d21e9841db45"
format_stamp: "Formatted at 2026-08-08 14:39:04 on dist-test-slave-p3t5"
server_key: "1a505d695cc6f8a5eb31dc0e81acb496"
server_key_iv: "c5c8ed44900d85e96b891061381f80cd"
server_key_version: "encryptionkey@0"
I20260808 14:39:04.540676 23902 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 14:39:04.543417 23917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:04.544936 23902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:39:04.545110 23902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "eb0b711778c147818c98d21e9841db45"
format_stamp: "Formatted at 2026-08-08 14:39:04 on dist-test-slave-p3t5"
server_key: "1a505d695cc6f8a5eb31dc0e81acb496"
server_key_iv: "c5c8ed44900d85e96b891061381f80cd"
server_key_version: "encryptionkey@0"
I20260808 14:39:04.545257 23902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:04.559427 23902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:04.564168 23902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:04.564389 23902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:04.565207 23902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:04.566388 23902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:04.566457 23902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:04.566524 23902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:04.566560 23902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:04.576840 23902 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:39411
I20260808 14:39:04.576856 24035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:39411 every 8 connection(s)
I20260808 14:39:04.577924 23902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:39:04.582472 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 23902
I20260808 14:39:04.582607 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:39:04.582892 20033 external_mini_cluster.cc:1428] Setting key 307a774376ecd28fc11bf624ab869ebc
Aug 08 14:39:04 dist-test-slave-p3t5 krb5kdc[23653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199944, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:04.593899 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:04.579955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:54129 (local address 127.19.144.126:46287)
0808 14:39:04.580115 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:04.580118 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:04.581076 (+ 958us) server_negotiation.cc:407] Connection header received
0808 14:39:04.582595 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:04.582598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:04.582661 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:04.582765 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:04.584217 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.584750 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.586032 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.586252 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.586403 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:04.589169 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:04.589176 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:04.589183 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:04.589219 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:04.591226 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:04.591941 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:04.591947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:04.591951 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:04.592061 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:04.592471 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:04.592477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:04.592480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:04.592757 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:04.592878 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:04.593517 (+ 639us) server_negotiation.cc:299] Negotiation successful
0808 14:39:04.593686 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
I20260808 14:39:04.594897 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:04.580342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46287 (local address 127.19.144.66:54129)
0808 14:39:04.580907 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:04.580955 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:04.582352 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:04.582972 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:04.583003 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:04.583528 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:04.584063 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:04.584078 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.584900 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.584903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:04.585896 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:04.585907 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.586449 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.586454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:04.586631 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:04.587342 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:04.587360 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:04.588970 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:04.591432 (+ 2462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:04.591439 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:04.591454 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:04.591815 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:04.592172 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:04.592174 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:04.592176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:04.592355 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:04.592933 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:04.592941 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:04.593235 (+ 294us) client_negotiation.cc:769] Sending connection context
0808 14:39:04.593423 (+ 188us) client_negotiation.cc:240] Negotiation successful
0808 14:39:04.593833 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":120,"threads_started":1}
I20260808 14:39:04.596539 24036 heartbeater.cc:347] Connected to a master server at 127.19.144.126:46287
I20260808 14:39:04.596871 24036 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:04.597669 24036 heartbeater.cc:511] Master 127.19.144.126:46287 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
I20260808 14:39:04.598989 23695 ts_manager.cc:194] Registered new tserver with Master: eb0b711778c147818c98d21e9841db45 (127.19.144.66:39411)
I20260808 14:39:04.599637 23695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:54129
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:39:04.617723 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46287
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:43107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:04.736496 24041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:04.736786 24041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:04.736831 24041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:04.741145 24041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:04.741225 24041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:04.741313 24041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:39:04.746745 24041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:43107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:04.747980 24041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:04.749137 24041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:04.759904 24046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:04.759894 24047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:04.759894 24049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:04.760149 24041 server_base.cc:1034] running on GCE node
I20260808 14:39:04.760577 24041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:04.761301 24041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:04.762496 24041 hybrid_clock.cc:648] HybridClock initialized: now 1786199944762453 us; error 55 us; skew 500 ppm
Aug 08 14:39:04 dist-test-slave-p3t5 krb5kdc[23653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199944, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:04.765513 24041 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:39:04.766585 24041 webserver.cc:460] Webserver started at http://127.19.144.67:38965/ using document root <none> and password file <none>
I20260808 14:39:04.767230 24041 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:04.767330 24041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:04.767582 24041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:04.769827 24041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "951bf01499104eeabac5376a2218e94c"
format_stamp: "Formatted at 2026-08-08 14:39:04 on dist-test-slave-p3t5"
server_key: "29c6f65b7e4b038826de085704499d85"
server_key_iv: "d11e39daaad5928d5190fbb8296b4e6f"
server_key_version: "encryptionkey@0"
I20260808 14:39:04.770605 24041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "951bf01499104eeabac5376a2218e94c"
format_stamp: "Formatted at 2026-08-08 14:39:04 on dist-test-slave-p3t5"
server_key: "29c6f65b7e4b038826de085704499d85"
server_key_iv: "d11e39daaad5928d5190fbb8296b4e6f"
server_key_version: "encryptionkey@0"
I20260808 14:39:04.774832 24041 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:39:04.777516 24056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:04.778856 24041 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:39:04.779006 24041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "951bf01499104eeabac5376a2218e94c"
format_stamp: "Formatted at 2026-08-08 14:39:04 on dist-test-slave-p3t5"
server_key: "29c6f65b7e4b038826de085704499d85"
server_key_iv: "d11e39daaad5928d5190fbb8296b4e6f"
server_key_version: "encryptionkey@0"
I20260808 14:39:04.779147 24041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:04.795194 24041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:04.798859 24041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:04.799099 24041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:04.799773 24041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:04.800724 24041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:04.800801 24041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:04.800894 24041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:04.800951 24041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:04.810632 24041 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:35841
I20260808 14:39:04.810676 24173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:35841 every 8 connection(s)
I20260808 14:39:04.811743 24041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:39:04.816056 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 24041
I20260808 14:39:04.816218 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:39:04.816509 20033 external_mini_cluster.cc:1428] Setting key 03ecdc71546129a20cf4227d2e63b7af
Aug 08 14:39:04 dist-test-slave-p3t5 krb5kdc[23653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199944, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:04.826610 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:04.814084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:51591 (local address 127.19.144.126:46287)
0808 14:39:04.814266 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:04.814271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:04.815172 (+ 901us) server_negotiation.cc:407] Connection header received
0808 14:39:04.816681 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:04.816686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:04.816747 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:04.816846 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:04.818593 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.819259 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.820418 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.820623 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.820699 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:04.823422 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:04.823425 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:04.823428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:04.823452 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:04.824742 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:04.825207 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:04.825210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:04.825211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:04.825288 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:04.825586 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:04.825588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:04.825590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:04.825829 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:04.825930 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:04.826385 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 14:39:04.826487 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 14:39:04.827497 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:04.814525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46287 (local address 127.19.144.67:51591)
0808 14:39:04.815008 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:04.815054 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:04.816445 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:04.817083 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:04.817092 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:04.817714 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:04.818437 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:04.818459 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.819382 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.819385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:04.820306 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:04.820316 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.820749 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.820752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:04.820900 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:04.821601 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:04.821669 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:04.823269 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:04.824846 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:04.824851 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:04.824863 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:04.825120 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:04.825367 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:04.825369 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:04.825370 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:04.825507 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:04.825939 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:04.825945 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:04.826177 (+ 232us) client_negotiation.cc:769] Sending connection context
0808 14:39:04.826334 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 14:39:04.826607 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":125,"threads_started":1}
I20260808 14:39:04.828809 24174 heartbeater.cc:347] Connected to a master server at 127.19.144.126:46287
I20260808 14:39:04.829098 24174 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:04.829721 24174 heartbeater.cc:511] Master 127.19.144.126:46287 requested a full tablet report, sending...
I20260808 14:39:04.830909 23695 ts_manager.cc:194] Registered new tserver with Master: 951bf01499104eeabac5376a2218e94c (127.19.144.67:35841)
I20260808 14:39:04.831494 23695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:51591
I20260808 14:39:04.842698 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 14:39:04 dist-test-slave-p3t5 krb5kdc[23653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 14:39:04 dist-test-slave-p3t5 krb5kdc[23653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.65@KRBTEST.COM
I20260808 14:39:04.872108 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:04.861575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38772 (local address 127.19.144.65:46263)
0808 14:39:04.861940 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:04.861947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:04.861973 (+ 26us) server_negotiation.cc:407] Connection header received
0808 14:39:04.862040 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:04.862046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:04.862255 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:04.862385 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:04.863151 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.863989 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.864909 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.865311 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.865410 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:04.868192 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:04.868205 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:04.868220 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:04.868250 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:04.870127 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:04.870626 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:04.870632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:04.870633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:04.870723 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:04.871037 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:04.871041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:04.871042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:04.871319 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:04.871563 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:04.871743 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 14:39:04.871928 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":165,"threads_started":1}
Aug 08 14:39:04 dist-test-slave-p3t5 krb5kdc[23653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:04.884223 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:04.875521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48330 (local address 127.19.144.126:46287)
0808 14:39:04.875685 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:04.875689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:04.875704 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:39:04.875831 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:04.875833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:04.875875 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:04.875943 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:04.876571 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.877111 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.878022 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:04.878235 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:04.878321 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:04.880871 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:04.880874 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:04.880877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:04.880906 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:04.882532 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:04.882974 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:04.882977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:04.882979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:04.883084 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:04.883432 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:04.883434 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:04.883435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:04.883660 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:04.883766 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:04.883994 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 14:39:04.884096 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 14:39:04.884721 23695 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48330
I20260808 14:39:04.885365 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 23762
I20260808 14:39:04.894002 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 23902
I20260808 14:39:04.902916 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 24041
I20260808 14:39:04.909991 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 23666
2026-08-08T14:39:04Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2259 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:39:04 dist-test-slave-p3t5 krb5kdc[24187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:39:04 dist-test-slave-p3t5 krb5kdc[24187](info): set up 2 sockets
Aug 08 14:39:04 dist-test-slave-p3t5 krb5kdc[24187](info): commencing operation
krb5kdc: starting...
W20260808 14:39:06.011765 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:39:06 dist-test-slave-p3t5 krb5kdc[24187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:39:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:39:06.153070 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:42017
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:42017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:06.273002 24201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:06.273404 24201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:06.273505 24201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:06.277855 24201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:06.277980 24201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:06.278017 24201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:06.278060 24201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:06.278095 24201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:39:06.283881 24201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:42017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:42017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:06.285187 24201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:06.286372 24201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:06.295392 24207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:06.295485 24208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:06.295387 24210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:06.295744 24201 server_base.cc:1034] running on GCE node
I20260808 14:39:06.296443 24201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:06.297719 24201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:06.298938 24201 hybrid_clock.cc:648] HybridClock initialized: now 1786199946298914 us; error 47 us; skew 500 ppm
Aug 08 14:39:06 dist-test-slave-p3t5 krb5kdc[24187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199946, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:06.302357 24201 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:39:06.303293 24201 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:06.303370 24201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:06.303568 24201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:06.305929 24201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "1719e871b5454258aa74fd3e1f6c5b04"
format_stamp: "Formatted at 2026-08-08 14:39:06 on dist-test-slave-p3t5"
server_key: "96db7636d0b4ef47be7a5238a2760a24"
server_key_iv: "6e967e07ad1efc17cad5d4d748eb51b7"
server_key_version: "encryptionkey@0"
I20260808 14:39:06.306560 24201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "1719e871b5454258aa74fd3e1f6c5b04"
format_stamp: "Formatted at 2026-08-08 14:39:06 on dist-test-slave-p3t5"
server_key: "96db7636d0b4ef47be7a5238a2760a24"
server_key_iv: "6e967e07ad1efc17cad5d4d748eb51b7"
server_key_version: "encryptionkey@0"
I20260808 14:39:06.311051 24201 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 14:39:06.313853 24217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:06.315286 24201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:06.315456 24201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "1719e871b5454258aa74fd3e1f6c5b04"
format_stamp: "Formatted at 2026-08-08 14:39:06 on dist-test-slave-p3t5"
server_key: "96db7636d0b4ef47be7a5238a2760a24"
server_key_iv: "6e967e07ad1efc17cad5d4d748eb51b7"
server_key_version: "encryptionkey@0"
I20260808 14:39:06.315593 24201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:06.350548 24201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:06.354382 24201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:06.354621 24201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:06.362186 24201 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:42017
I20260808 14:39:06.362505 24270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:42017 every 8 connection(s)
I20260808 14:39:06.362973 24201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:06.365976 24271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:06.371639 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 24201
I20260808 14:39:06.371793 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:39:06.372116 20033 external_mini_cluster.cc:1428] Setting key bcf15c1cfa9ec56d94507812885c200e
I20260808 14:39:06.372265 24271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04: Bootstrap starting.
I20260808 14:39:06.375108 24271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:06.376120 24271 log.cc:824] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:06.378548 24271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04: No bootstrap required, opened a new log
Aug 08 14:39:06 dist-test-slave-p3t5 krb5kdc[24187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:06.383222 24271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1719e871b5454258aa74fd3e1f6c5b04" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 42017 } }
I20260808 14:39:06.383520 24271 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:06.383637 24271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1719e871b5454258aa74fd3e1f6c5b04, State: Initialized, Role: FOLLOWER
I20260808 14:39:06.384156 24271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1719e871b5454258aa74fd3e1f6c5b04" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 42017 } }
I20260808 14:39:06.384325 24271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:06.384425 24271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:06.384552 24271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:06.385536 24271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1719e871b5454258aa74fd3e1f6c5b04" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 42017 } }
I20260808 14:39:06.385941 24271 leader_election.cc:302] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1719e871b5454258aa74fd3e1f6c5b04; no voters:
I20260808 14:39:06.386348 24271 leader_election.cc:288] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:39:06.386444 24276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:06.386734 24276 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [term 1 LEADER]: Becoming Leader. State: Replica: 1719e871b5454258aa74fd3e1f6c5b04, State: Running, Role: LEADER
I20260808 14:39:06.387135 24276 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1719e871b5454258aa74fd3e1f6c5b04" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 42017 } }
I20260808 14:39:06.387933 24271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:06.388113 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:06.373565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57926 (local address 127.19.144.126:42017)
0808 14:39:06.374095 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:06.374112 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:06.374152 (+ 40us) server_negotiation.cc:407] Connection header received
0808 14:39:06.374841 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:06.374859 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:06.375216 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:06.375598 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:06.376573 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:06.377825 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:06.379020 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:06.379503 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:06.379623 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:06.382601 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:06.382607 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:06.382623 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:06.382655 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:06.384743 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:06.385235 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:06.385242 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:06.385248 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:06.385391 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:06.385761 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:06.385765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:06.385766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:06.386284 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:06.386496 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:06.386832 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 14:39:06.387198 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":167,"threads_started":1}
I20260808 14:39:06.388440 24276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1719e871b5454258aa74fd3e1f6c5b04. Latest consensus state: current_term: 1 leader_uuid: "1719e871b5454258aa74fd3e1f6c5b04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1719e871b5454258aa74fd3e1f6c5b04" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 42017 } } }
I20260808 14:39:06.388563 24276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:06.388798 24276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1719e871b5454258aa74fd3e1f6c5b04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1719e871b5454258aa74fd3e1f6c5b04" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 42017 } } }
I20260808 14:39:06.388921 24276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:06.389117 24280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:06.392676 24280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:06.399087 24280 catalog_manager.cc:1347] Generated new cluster ID: 905608f6af34476f8d1ee2358054fe5b
I20260808 14:39:06.399183 24280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:06.410851 24280 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:39:06.412286 24280 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:06.422700 24280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04: Generated new TSK 0
I20260808 14:39:06.423617 24280 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:39:06.466778 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:42017
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:06.588203 24297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:06.588562 24297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:06.588645 24297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:06.593093 24297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:06.593182 24297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:06.593274 24297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:39:06.598941 24297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:42017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:06.600200 24297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:06.601337 24297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:06.612007 24306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:06.612078 24297 server_base.cc:1034] running on GCE node
W20260808 14:39:06.612001 24303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:06.611995 24304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:06.612843 24297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:06.613596 24297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:06.614876 24297 hybrid_clock.cc:648] HybridClock initialized: now 1786199946614842 us; error 53 us; skew 500 ppm
Aug 08 14:39:06 dist-test-slave-p3t5 krb5kdc[24187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199946, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:06.618129 24297 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:39:06.618947 24297 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:06.619019 24297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:06.619207 24297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:06.621536 24297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "deea4595b35b4e39ba640478ca997e3a"
format_stamp: "Formatted at 2026-08-08 14:39:06 on dist-test-slave-p3t5"
server_key: "4ab2a6b8a309bcb97debac5e4b175a95"
server_key_iv: "4c87e060d299a8ee960d9fdff9c9ff10"
server_key_version: "encryptionkey@0"
I20260808 14:39:06.622196 24297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "deea4595b35b4e39ba640478ca997e3a"
format_stamp: "Formatted at 2026-08-08 14:39:06 on dist-test-slave-p3t5"
server_key: "4ab2a6b8a309bcb97debac5e4b175a95"
server_key_iv: "4c87e060d299a8ee960d9fdff9c9ff10"
server_key_version: "encryptionkey@0"
I20260808 14:39:06.626310 24297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:39:06.628983 24312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:06.630491 24297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:06.630641 24297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "deea4595b35b4e39ba640478ca997e3a"
format_stamp: "Formatted at 2026-08-08 14:39:06 on dist-test-slave-p3t5"
server_key: "4ab2a6b8a309bcb97debac5e4b175a95"
server_key_iv: "4c87e060d299a8ee960d9fdff9c9ff10"
server_key_version: "encryptionkey@0"
I20260808 14:39:06.630770 24297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:06.648409 24297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:06.652074 24297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:06.652298 24297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:06.652954 24297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:06.653913 24297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:06.653965 24297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:06.654032 24297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:06.654064 24297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:06.663151 24297 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:43395
I20260808 14:39:06.663177 24428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:43395 every 8 connection(s)
I20260808 14:39:06.663832 24297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:39:06.665987 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 24297
I20260808 14:39:06.666115 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:39:06.666431 20033 external_mini_cluster.cc:1428] Setting key 60988c928923969357c18674613d70bf
Aug 08 14:39:06 dist-test-slave-p3t5 krb5kdc[24187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199946, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:39:06.678767 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:06.665610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:57139 (local address 127.19.144.126:42017)
0808 14:39:06.665812 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:06.665819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:06.667300 (+ 1481us) server_negotiation.cc:407] Connection header received
0808 14:39:06.668180 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:06.668184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:06.668236 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:06.668322 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:06.669566 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:06.670483 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:06.671724 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:06.672165 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:06.672295 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:06.674847 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:06.674854 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:06.674859 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:06.674892 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:06.676632 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:06.677148 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:06.677152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:06.677154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:06.677255 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:06.677602 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:06.677605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:06.677607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:06.677900 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:06.678009 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:06.678471 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 14:39:06.678625 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 14:39:06.679694 24431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:06.666112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:42017 (local address 127.19.144.65:57139)
0808 14:39:06.667135 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:06.667202 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:06.667958 (+ 756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:06.668524 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:06.668531 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:06.668919 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:06.669401 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:06.669415 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:06.670652 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:06.670656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:06.671572 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:06.671583 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:06.672319 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:06.672322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:06.672468 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:06.673115 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:06.673131 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:06.674670 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:06.676752 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:06.676757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:06.676769 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:06.677033 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:06.677352 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:06.677354 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:06.677355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:06.677492 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:06.678012 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:06.678017 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:06.678250 (+ 233us) client_negotiation.cc:769] Sending connection context
0808 14:39:06.678422 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 14:39:06.678727 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":145,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
I20260808 14:39:06.681190 24429 heartbeater.cc:347] Connected to a master server at 127.19.144.126:42017
I20260808 14:39:06.681468 24429 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:06.682024 24429 heartbeater.cc:511] Master 127.19.144.126:42017 requested a full tablet report, sending...
I20260808 14:39:06.683815 24235 ts_manager.cc:194] Registered new tserver with Master: deea4595b35b4e39ba640478ca997e3a (127.19.144.65:43395)
I20260808 14:39:06.685523 24235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:57139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:39:06.698828 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:42017
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:06.818251 24434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:06.818563 24434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:06.818614 24434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:06.822993 24434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:06.823068 24434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:06.823133 24434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:39:06.828670 24434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:42017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:06.829885 24434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:06.830956 24434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:06.841169 24442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:06.841197 24440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:06.841352 24444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:06.841250 24434 server_base.cc:1034] running on GCE node
I20260808 14:39:06.841898 24434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:06.842779 24434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:06.843993 24434 hybrid_clock.cc:648] HybridClock initialized: now 1786199946843954 us; error 49 us; skew 500 ppm
Aug 08 14:39:06 dist-test-slave-p3t5 krb5kdc[24187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199946, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:06.851691 24434 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:39:06.852593 24434 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:06.852697 24434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:06.852962 24434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:06.855237 24434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "a79348a2e91249a59b6006095eeffd18"
format_stamp: "Formatted at 2026-08-08 14:39:06 on dist-test-slave-p3t5"
server_key: "7ae37ab18b2e55c5f5ec14cf57569270"
server_key_iv: "9b4759d45725975a3a8dbc3d679d0908"
server_key_version: "encryptionkey@0"
I20260808 14:39:06.855851 24434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "a79348a2e91249a59b6006095eeffd18"
format_stamp: "Formatted at 2026-08-08 14:39:06 on dist-test-slave-p3t5"
server_key: "7ae37ab18b2e55c5f5ec14cf57569270"
server_key_iv: "9b4759d45725975a3a8dbc3d679d0908"
server_key_version: "encryptionkey@0"
I20260808 14:39:06.860067 24434 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:39:06.862751 24450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:06.864125 24434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:06.864282 24434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "a79348a2e91249a59b6006095eeffd18"
format_stamp: "Formatted at 2026-08-08 14:39:06 on dist-test-slave-p3t5"
server_key: "7ae37ab18b2e55c5f5ec14cf57569270"
server_key_iv: "9b4759d45725975a3a8dbc3d679d0908"
server_key_version: "encryptionkey@0"
I20260808 14:39:06.864408 24434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:06.893283 24434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:06.896819 24434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:06.897064 24434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:06.897792 24434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:06.898739 24434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:06.898823 24434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:06.898896 24434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:06.898945 24434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:06.908551 24434 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:44585
I20260808 14:39:06.908573 24567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:44585 every 8 connection(s)
I20260808 14:39:06.909281 24434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:39:06.916970 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 24434
I20260808 14:39:06.917104 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:39:06.917446 20033 external_mini_cluster.cc:1428] Setting key 50c9509ba1047fefdfc63ee57d7cb85a
Aug 08 14:39:06 dist-test-slave-p3t5 krb5kdc[24187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199946, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:06.924412 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:06.911250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:53141 (local address 127.19.144.126:42017)
0808 14:39:06.911407 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:06.911410 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:06.912629 (+ 1219us) server_negotiation.cc:407] Connection header received
0808 14:39:06.913476 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:06.913479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:06.913529 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:06.913607 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:06.914989 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:06.915543 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:06.916733 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:06.917080 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:06.917198 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:06.920247 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:06.920254 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:06.920260 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:06.920297 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:06.921981 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:06.922630 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:06.922634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:06.922636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:06.922729 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:06.923184 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:06.923187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:06.923189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:06.923406 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:06.923523 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:06.924093 (+ 570us) server_negotiation.cc:299] Negotiation successful
0808 14:39:06.924227 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:39:06.925383 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:06.911636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:42017 (local address 127.19.144.66:53141)
0808 14:39:06.912465 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:06.912509 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:06.913284 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:06.913879 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:06.913892 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:06.914287 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:06.914809 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:06.914823 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:06.915664 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:06.915666 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:06.916575 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:06.916585 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:06.917262 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:06.917265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:06.917418 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:06.918195 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:06.918221 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:06.919980 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:06.922130 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:06.922136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:06.922155 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:06.922529 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:06.922853 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:06.922856 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:06.922859 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:06.923079 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:06.923522 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:06.923529 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:06.923827 (+ 298us) client_negotiation.cc:769] Sending connection context
0808 14:39:06.924047 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 14:39:06.924416 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":118,"threads_started":1}
I20260808 14:39:06.926697 24568 heartbeater.cc:347] Connected to a master server at 127.19.144.126:42017
I20260808 14:39:06.927033 24568 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:06.927660 24568 heartbeater.cc:511] Master 127.19.144.126:42017 requested a full tablet report, sending...
I20260808 14:39:06.929008 24235 ts_manager.cc:194] Registered new tserver with Master: a79348a2e91249a59b6006095eeffd18 (127.19.144.66:44585)
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
I20260808 14:39:06.929749 24235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:53141
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:39:06.952916 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:42017
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:07.078339 24574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:07.078625 24574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:07.078673 24574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:07.082896 24574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:07.082971 24574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:07.083070 24574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:39:07.088670 24574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:42017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:07.090010 24574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:07.091259 24574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:07.101584 24579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:07.101585 24583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:07.101584 24580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:07.102020 24574 server_base.cc:1034] running on GCE node
I20260808 14:39:07.102556 24574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:07.103298 24574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:07.104501 24574 hybrid_clock.cc:648] HybridClock initialized: now 1786199947104481 us; error 32 us; skew 500 ppm
Aug 08 14:39:07 dist-test-slave-p3t5 krb5kdc[24187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199947, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:07.107646 24574 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:39:07.108489 24574 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:07.108595 24574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:07.108860 24574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:07.111183 24574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "a709af21ccb54286b60b3af701f33674"
format_stamp: "Formatted at 2026-08-08 14:39:07 on dist-test-slave-p3t5"
server_key: "3b9cb86d79b2003de0b6c54daaadfe5d"
server_key_iv: "99756aa7b31030bf88a60b60b942540f"
server_key_version: "encryptionkey@0"
I20260808 14:39:07.111825 24574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "a709af21ccb54286b60b3af701f33674"
format_stamp: "Formatted at 2026-08-08 14:39:07 on dist-test-slave-p3t5"
server_key: "3b9cb86d79b2003de0b6c54daaadfe5d"
server_key_iv: "99756aa7b31030bf88a60b60b942540f"
server_key_version: "encryptionkey@0"
I20260808 14:39:07.116025 24574 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260808 14:39:07.118781 24589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:07.120313 24574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:07.120478 24574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "a709af21ccb54286b60b3af701f33674"
format_stamp: "Formatted at 2026-08-08 14:39:07 on dist-test-slave-p3t5"
server_key: "3b9cb86d79b2003de0b6c54daaadfe5d"
server_key_iv: "99756aa7b31030bf88a60b60b942540f"
server_key_version: "encryptionkey@0"
I20260808 14:39:07.120613 24574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:07.132660 24574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:07.136466 24574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:07.136687 24574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:07.137331 24574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:07.138278 24574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:07.138355 24574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:07.138429 24574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:07.138484 24574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:07.147516 24574 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:41603
I20260808 14:39:07.147538 24707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:41603 every 8 connection(s)
I20260808 14:39:07.148200 24574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:39:07.152230 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 24574
I20260808 14:39:07.152347 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:39:07.152711 20033 external_mini_cluster.cc:1428] Setting key 11b6924753982a17ca9cef678087d477
Aug 08 14:39:07 dist-test-slave-p3t5 krb5kdc[24187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199947, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:07.163160 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.150187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:53127 (local address 127.19.144.126:42017)
0808 14:39:07.150361 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:07.150365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:07.151551 (+ 1186us) server_negotiation.cc:407] Connection header received
0808 14:39:07.152498 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:07.152502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:07.152575 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:07.152674 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:07.154179 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.154726 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.155944 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.156173 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.156292 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:07.159154 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:07.159157 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:07.159159 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:07.159213 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:07.160816 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:07.161424 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:07.161427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:07.161428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:07.161514 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:07.161946 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:07.161950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:07.161952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:07.162164 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:07.162281 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:07.162842 (+ 561us) server_negotiation.cc:299] Negotiation successful
0808 14:39:07.162986 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 14:39:07.164062 24710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.150508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:42017 (local address 127.19.144.67:53127)
0808 14:39:07.151371 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:07.151423 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:07.152227 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:07.152881 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:07.152897 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:07.153323 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:07.154028 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:07.154045 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.154862 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.154865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:07.155802 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:07.155812 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.156340 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.156345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:07.156522 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:07.157187 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:07.157204 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:07.158951 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:07.160968 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:07.160975 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:07.160992 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:07.161288 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:07.161646 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:07.161651 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:07.161653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:07.161821 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:07.162281 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:07.162290 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:07.162583 (+ 293us) client_negotiation.cc:769] Sending connection context
0808 14:39:07.162784 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 14:39:07.163167 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":121,"threads_started":1}
I20260808 14:39:07.165292 24708 heartbeater.cc:347] Connected to a master server at 127.19.144.126:42017
I20260808 14:39:07.165587 24708 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:07.166189 24708 heartbeater.cc:511] Master 127.19.144.126:42017 requested a full tablet report, sending...
I20260808 14:39:07.167338 24235 ts_manager.cc:194] Registered new tserver with Master: a709af21ccb54286b60b3af701f33674 (127.19.144.67:41603)
I20260808 14:39:07.167981 24235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:53127
I20260808 14:39:07.179495 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:39:07.189689 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.181645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57928 (local address 127.19.144.126:42017)
0808 14:39:07.181816 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:07.181823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:07.182006 (+ 183us) server_negotiation.cc:407] Connection header received
0808 14:39:07.182174 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:07.182177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:07.182246 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:07.182334 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:07.183098 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.183641 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.184502 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.184732 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.184846 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:07.186386 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:07.186390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:07.186392 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:07.186425 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:07.187773 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:07.188259 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:07.188263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:07.188264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:07.188401 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:07.188698 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:07.188702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:07.188704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:07.188960 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:07.189104 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:07.189360 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 14:39:07.189487 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:39:07.192675 24235 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:39:07.195948 24235 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:39:07.212423 24720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.205726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:43395 (local address 127.0.0.1:51126)
0808 14:39:07.206194 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:07.206248 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:07.206392 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:07.207463 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:07.207470 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:07.207504 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:07.207712 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:07.207724 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.209265 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.209272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:07.210665 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:07.210676 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.211813 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.211817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:07.211928 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:07.211975 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 14:39:07.212104 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 14:39:07.212249 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":100,"threads_started":1}
I20260808 14:39:07.212476 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.205856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51126 (local address 127.19.144.65:43395)
0808 14:39:07.206755 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:07.206766 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:07.206794 (+ 28us) server_negotiation.cc:407] Connection header received
0808 14:39:07.206887 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:07.206893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:07.207134 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:07.207924 (+ 790us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:07.208091 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.209131 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.210802 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.211665 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.211800 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:07.211980 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:07.212164 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 14:39:07.212288 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":59,"threads_started":1}
I20260808 14:39:07.212970 24719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.205766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:41603 (local address 127.0.0.1:43710)
0808 14:39:07.207030 (+ 1264us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:07.207046 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:07.207144 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:07.207764 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:07.207770 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:07.207796 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:07.207930 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:07.207936 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.209788 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.209792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:07.210984 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:07.210995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.212447 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.212451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:07.212604 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:07.212629 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 14:39:07.212705 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 14:39:07.212782 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1201,"mutex_wait_us":32,"thread_start_us":130,"threads_started":1}
I20260808 14:39:07.213173 24724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.207009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40992 (local address 127.19.144.66:44585)
0808 14:39:07.207493 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:07.207512 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:07.207925 (+ 413us) server_negotiation.cc:407] Connection header received
0808 14:39:07.208038 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:07.208042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:07.208230 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:07.208396 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:07.208671 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.209817 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.211400 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.212121 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.212215 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:07.212358 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:07.212870 (+ 512us) server_negotiation.cc:299] Negotiation successful
0808 14:39:07.212976 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":153,"threads_started":1}
I20260808 14:39:07.213374 24723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.206699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:44585 (local address 127.0.0.1:40992)
0808 14:39:07.207714 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:07.207728 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:07.207830 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:07.208420 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:07.208423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:07.208445 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:07.208537 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:07.208543 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.209964 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.209967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:07.211277 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:07.211285 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.212421 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.212425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:07.212548 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:07.212583 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 14:39:07.212661 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 14:39:07.212741 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":935,"mutex_wait_us":3,"thread_start_us":101,"threads_started":1}
I20260808 14:39:07.213438 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.206593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43710 (local address 127.19.144.67:41603)
0808 14:39:07.207093 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:07.207105 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:07.207128 (+ 23us) server_negotiation.cc:407] Connection header received
0808 14:39:07.207360 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:07.207368 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:07.207585 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:07.207772 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:07.208058 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.209587 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.211147 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.212109 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.212227 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:07.212372 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:07.213172 (+ 800us) server_negotiation.cc:299] Negotiation successful
0808 14:39:07.213281 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":155,"threads_started":1}
I20260808 14:39:07.216264 24362 tablet_service.cc:1467] Processing CreateTablet for tablet 4f4153efd72541c78d3cf8afae21c85f (DEFAULT_TABLE table=test-table [id=6308b36c5b794eaea9b0ea9cd1e7bdf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:07.216539 24639 tablet_service.cc:1467] Processing CreateTablet for tablet 4f4153efd72541c78d3cf8afae21c85f (DEFAULT_TABLE table=test-table [id=6308b36c5b794eaea9b0ea9cd1e7bdf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:07.216539 24500 tablet_service.cc:1467] Processing CreateTablet for tablet 4f4153efd72541c78d3cf8afae21c85f (DEFAULT_TABLE table=test-table [id=6308b36c5b794eaea9b0ea9cd1e7bdf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:07.217504 24362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f4153efd72541c78d3cf8afae21c85f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:07.217607 24500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f4153efd72541c78d3cf8afae21c85f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:07.217640 24639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f4153efd72541c78d3cf8afae21c85f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:07.225291 24726 tablet_bootstrap.cc:492] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a: Bootstrap starting.
I20260808 14:39:07.226028 24725 tablet_bootstrap.cc:492] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18: Bootstrap starting.
I20260808 14:39:07.226481 24727 tablet_bootstrap.cc:492] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674: Bootstrap starting.
I20260808 14:39:07.228368 24727 tablet_bootstrap.cc:654] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:07.228369 24725 tablet_bootstrap.cc:654] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:07.229552 24727 log.cc:824] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:07.229622 24725 log.cc:824] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:07.231855 24727 tablet_bootstrap.cc:492] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674: No bootstrap required, opened a new log
I20260808 14:39:07.232000 24725 tablet_bootstrap.cc:492] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18: No bootstrap required, opened a new log
I20260808 14:39:07.232165 24727 ts_tablet_manager.cc:1397] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:39:07.232261 24725 ts_tablet_manager.cc:1397] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260808 14:39:07.232620 24726 tablet_bootstrap.cc:654] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:07.233530 24726 log.cc:824] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:07.236042 24726 tablet_bootstrap.cc:492] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a: No bootstrap required, opened a new log
I20260808 14:39:07.236325 24726 ts_tablet_manager.cc:1397] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.006s
I20260808 14:39:07.236299 24727 raft_consensus.cc:348] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deea4595b35b4e39ba640478ca997e3a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43395 } } peers { permanent_uuid: "a709af21ccb54286b60b3af701f33674" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41603 } } peers { permanent_uuid: "a79348a2e91249a59b6006095eeffd18" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44585 } }
I20260808 14:39:07.236290 24725 raft_consensus.cc:348] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deea4595b35b4e39ba640478ca997e3a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43395 } } peers { permanent_uuid: "a709af21ccb54286b60b3af701f33674" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41603 } } peers { permanent_uuid: "a79348a2e91249a59b6006095eeffd18" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44585 } }
I20260808 14:39:07.236613 24725 raft_consensus.cc:374] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:07.236699 24725 raft_consensus.cc:729] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a79348a2e91249a59b6006095eeffd18, State: Initialized, Role: FOLLOWER
I20260808 14:39:07.236770 24727 raft_consensus.cc:374] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:07.236941 24727 raft_consensus.cc:729] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a709af21ccb54286b60b3af701f33674, State: Initialized, Role: FOLLOWER
I20260808 14:39:07.237317 24725 consensus_queue.cc:260] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deea4595b35b4e39ba640478ca997e3a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43395 } } peers { permanent_uuid: "a709af21ccb54286b60b3af701f33674" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41603 } } peers { permanent_uuid: "a79348a2e91249a59b6006095eeffd18" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44585 } }
I20260808 14:39:07.237531 24727 consensus_queue.cc:260] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deea4595b35b4e39ba640478ca997e3a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43395 } } peers { permanent_uuid: "a709af21ccb54286b60b3af701f33674" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41603 } } peers { permanent_uuid: "a79348a2e91249a59b6006095eeffd18" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44585 } }
I20260808 14:39:07.238405 24568 heartbeater.cc:503] Master 127.19.144.126:42017 was elected leader, sending a full tablet report...
I20260808 14:39:07.238795 24725 ts_tablet_manager.cc:1428] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260808 14:39:07.238787 24727 ts_tablet_manager.cc:1428] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 14:39:07.239707 24708 heartbeater.cc:503] Master 127.19.144.126:42017 was elected leader, sending a full tablet report...
I20260808 14:39:07.241146 24726 raft_consensus.cc:348] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deea4595b35b4e39ba640478ca997e3a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43395 } } peers { permanent_uuid: "a709af21ccb54286b60b3af701f33674" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41603 } } peers { permanent_uuid: "a79348a2e91249a59b6006095eeffd18" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44585 } }
I20260808 14:39:07.241472 24726 raft_consensus.cc:374] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:07.241600 24726 raft_consensus.cc:729] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: deea4595b35b4e39ba640478ca997e3a, State: Initialized, Role: FOLLOWER
I20260808 14:39:07.242165 24726 consensus_queue.cc:260] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deea4595b35b4e39ba640478ca997e3a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43395 } } peers { permanent_uuid: "a709af21ccb54286b60b3af701f33674" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41603 } } peers { permanent_uuid: "a79348a2e91249a59b6006095eeffd18" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44585 } }
I20260808 14:39:07.242996 24429 heartbeater.cc:503] Master 127.19.144.126:42017 was elected leader, sending a full tablet report...
I20260808 14:39:07.243579 24726 ts_tablet_manager.cc:1428] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260808 14:39:07.267396 24731 raft_consensus.cc:482] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:39:07.267719 24731 raft_consensus.cc:504] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deea4595b35b4e39ba640478ca997e3a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43395 } } peers { permanent_uuid: "a709af21ccb54286b60b3af701f33674" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41603 } } peers { permanent_uuid: "a79348a2e91249a59b6006095eeffd18" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44585 } }
I20260808 14:39:07.269204 24731 leader_election.cc:288] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers deea4595b35b4e39ba640478ca997e3a (127.19.144.65:43395), a709af21ccb54286b60b3af701f33674 (127.19.144.67:41603)
I20260808 14:39:07.273573 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.269452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:43395 (local address 127.19.144.66:37923)
0808 14:39:07.269716 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:07.269734 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:07.269838 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:07.270453 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:07.270457 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:07.270478 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:07.270629 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:07.270636 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.271486 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.271489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:07.272362 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:07.272372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.273100 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.273104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:07.273227 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:07.273250 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 14:39:07.273346 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 14:39:07.273407 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260808 14:39:07.273661 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.270017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:37923 (local address 127.19.144.65:43395)
0808 14:39:07.270170 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:07.270175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:07.270192 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:39:07.270257 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:07.270261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:07.270318 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:07.270400 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:07.270765 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.271350 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.272504 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.272967 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.273067 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:07.273111 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:07.273368 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 14:39:07.273470 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 14:39:07.273727 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.269682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:42543 (local address 127.19.144.67:41603)
0808 14:39:07.269949 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:07.269954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:07.270316 (+ 362us) server_negotiation.cc:407] Connection header received
0808 14:39:07.270420 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:07.270426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:07.270486 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:07.270577 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:07.270955 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.271501 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.272627 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.273102 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.273223 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:07.273275 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:07.273501 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 14:39:07.273573 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260808 14:39:07.273738 24734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.269452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:41603 (local address 127.19.144.66:42543)
0808 14:39:07.270173 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:07.270189 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:07.270302 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:07.270601 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:07.270604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:07.270620 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:07.270754 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:07.270760 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.271622 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.271624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:07.272492 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:07.272502 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.273225 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.273228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:07.273355 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:07.273382 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 14:39:07.273459 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 14:39:07.273523 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"mutex_wait_us":174,"thread_start_us":198,"threads_started":1}
I20260808 14:39:07.274444 24383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f4153efd72541c78d3cf8afae21c85f" candidate_uuid: "a79348a2e91249a59b6006095eeffd18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "deea4595b35b4e39ba640478ca997e3a" is_pre_election: true
I20260808 14:39:07.274444 24659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f4153efd72541c78d3cf8afae21c85f" candidate_uuid: "a79348a2e91249a59b6006095eeffd18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a709af21ccb54286b60b3af701f33674" is_pre_election: true
I20260808 14:39:07.274792 24659 raft_consensus.cc:2454] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a79348a2e91249a59b6006095eeffd18 in term 0.
I20260808 14:39:07.274792 24383 raft_consensus.cc:2454] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a79348a2e91249a59b6006095eeffd18 in term 0.
I20260808 14:39:07.275367 24454 leader_election.cc:302] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a709af21ccb54286b60b3af701f33674, a79348a2e91249a59b6006095eeffd18, deea4595b35b4e39ba640478ca997e3a; no voters:
I20260808 14:39:07.275689 24731 raft_consensus.cc:2781] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:39:07.275781 24731 raft_consensus.cc:482] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:39:07.275825 24731 raft_consensus.cc:3037] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:07.276919 24731 raft_consensus.cc:504] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deea4595b35b4e39ba640478ca997e3a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43395 } } peers { permanent_uuid: "a709af21ccb54286b60b3af701f33674" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41603 } } peers { permanent_uuid: "a79348a2e91249a59b6006095eeffd18" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44585 } }
I20260808 14:39:07.277266 24731 leader_election.cc:288] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [CANDIDATE]: Term 1 election: Requested vote from peers deea4595b35b4e39ba640478ca997e3a (127.19.144.65:43395), a709af21ccb54286b60b3af701f33674 (127.19.144.67:41603)
I20260808 14:39:07.277693 24659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f4153efd72541c78d3cf8afae21c85f" candidate_uuid: "a79348a2e91249a59b6006095eeffd18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a709af21ccb54286b60b3af701f33674"
I20260808 14:39:07.277688 24383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f4153efd72541c78d3cf8afae21c85f" candidate_uuid: "a79348a2e91249a59b6006095eeffd18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "deea4595b35b4e39ba640478ca997e3a"
I20260808 14:39:07.277853 24659 raft_consensus.cc:3037] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:07.277868 24383 raft_consensus.cc:3037] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:07.278956 24659 raft_consensus.cc:2454] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a79348a2e91249a59b6006095eeffd18 in term 1.
I20260808 14:39:07.278999 24383 raft_consensus.cc:2454] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a79348a2e91249a59b6006095eeffd18 in term 1.
I20260808 14:39:07.279305 24454 leader_election.cc:302] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a709af21ccb54286b60b3af701f33674, a79348a2e91249a59b6006095eeffd18; no voters:
I20260808 14:39:07.279520 24731 raft_consensus.cc:2781] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:07.279778 24731 raft_consensus.cc:686] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [term 1 LEADER]: Becoming Leader. State: Replica: a79348a2e91249a59b6006095eeffd18, State: Running, Role: LEADER
I20260808 14:39:07.280159 24731 consensus_queue.cc:237] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deea4595b35b4e39ba640478ca997e3a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43395 } } peers { permanent_uuid: "a709af21ccb54286b60b3af701f33674" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41603 } } peers { permanent_uuid: "a79348a2e91249a59b6006095eeffd18" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44585 } }
I20260808 14:39:07.283679 24232 catalog_manager.cc:5495] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 reported cstate change: term changed from 0 to 1, leader changed from <none> to a79348a2e91249a59b6006095eeffd18 (127.19.144.66). New cstate: current_term: 1 leader_uuid: "a79348a2e91249a59b6006095eeffd18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deea4595b35b4e39ba640478ca997e3a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a709af21ccb54286b60b3af701f33674" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a79348a2e91249a59b6006095eeffd18" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44585 } health_report { overall_health: HEALTHY } } }
I20260808 14:39:07.314957 24724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.311035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41000 (local address 127.19.144.66:44585)
0808 14:39:07.311177 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:07.311180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:07.311197 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:39:07.311324 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:07.311326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:07.311390 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:07.311498 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:39:07.311822 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.312382 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.313246 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:07.313465 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:07.313568 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:07.314071 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:39:07.314532 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:39:07.314649 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:07.314719 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 14:39:07.314814 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:39:07.321878 24383 raft_consensus.cc:1261] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a [term 1 FOLLOWER]: Refusing update from remote peer a79348a2e91249a59b6006095eeffd18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:07.321873 24659 raft_consensus.cc:1261] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674 [term 1 FOLLOWER]: Refusing update from remote peer a79348a2e91249a59b6006095eeffd18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:07.322674 24731 consensus_queue.cc:1035] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a709af21ccb54286b60b3af701f33674" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 41603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:07.322911 24735 consensus_queue.cc:1035] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "deea4595b35b4e39ba640478ca997e3a" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:07.331782 24739 mvcc.cc:205] Tried to move back new op lower bound from 7316274984221003776 to 7316274984062541824. Current Snapshot: MvccSnapshot[applied={T|T < 7316274984221003776}]
I20260808 14:39:07.333216 24740 mvcc.cc:205] Tried to move back new op lower bound from 7316274984221003776 to 7316274984062541824. Current Snapshot: MvccSnapshot[applied={T|T < 7316274984221003776}]
I20260808 14:39:07.339614 24233 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:07.339814 24233 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:07.342434 24233 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1719e871b5454258aa74fd3e1f6c5b04: Sending DeleteTablet for 3 replicas of tablet 4f4153efd72541c78d3cf8afae21c85f
I20260808 14:39:07.343216 24362 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f4153efd72541c78d3cf8afae21c85f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:07 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:51126
I20260808 14:39:07.343312 24639 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f4153efd72541c78d3cf8afae21c85f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:07 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:43710
I20260808 14:39:07.343408 24500 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f4153efd72541c78d3cf8afae21c85f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:07 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:40992
I20260808 14:39:07.343953 24751 tablet_replica.cc:330] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18: stopping tablet replica
I20260808 14:39:07.343954 24749 tablet_replica.cc:330] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a: stopping tablet replica
I20260808 14:39:07.344290 24749 raft_consensus.cc:2229] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:07.344309 24751 raft_consensus.cc:2229] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:39:07.344641 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 24297
I20260808 14:39:07.344712 24749 raft_consensus.cc:2258] T 4f4153efd72541c78d3cf8afae21c85f P deea4595b35b4e39ba640478ca997e3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:07.344784 24751 raft_consensus.cc:2258] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:07.345345 24750 tablet_replica.cc:330] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674: stopping tablet replica
I20260808 14:39:07.345642 24750 raft_consensus.cc:2229] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:07.345924 24750 raft_consensus.cc:2258] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:07.355286 24751 ts_tablet_manager.cc:1905] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:07.355286 24750 ts_tablet_manager.cc:1905] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:07.358263 24751 ts_tablet_manager.cc:1918] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:07.358263 24750 ts_tablet_manager.cc:1918] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:07.358400 24750 log.cc:1196] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/4f4153efd72541c78d3cf8afae21c85f
I20260808 14:39:07.358563 24751 log.cc:1196] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/4f4153efd72541c78d3cf8afae21c85f
I20260808 14:39:07.358774 24750 ts_tablet_manager.cc:1939] T 4f4153efd72541c78d3cf8afae21c85f P a709af21ccb54286b60b3af701f33674: Deleting consensus metadata
I20260808 14:39:07.359090 24751 ts_tablet_manager.cc:1939] T 4f4153efd72541c78d3cf8afae21c85f P a79348a2e91249a59b6006095eeffd18: Deleting consensus metadata
I20260808 14:39:07.360307 24219 catalog_manager.cc:4841] TS a79348a2e91249a59b6006095eeffd18 (127.19.144.66:44585): tablet 4f4153efd72541c78d3cf8afae21c85f (table test-table [id=6308b36c5b794eaea9b0ea9cd1e7bdf1]) successfully deleted
I20260808 14:39:07.360301 24220 catalog_manager.cc:4841] TS a709af21ccb54286b60b3af701f33674 (127.19.144.67:41603): tablet 4f4153efd72541c78d3cf8afae21c85f (table test-table [id=6308b36c5b794eaea9b0ea9cd1e7bdf1]) successfully deleted
W20260808 14:39:07.361548 24221 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.144.65:43395 (error 108)
I20260808 14:39:07.362664 24723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.362358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:43395 (local address 127.0.0.1:51138)
0808 14:39:07.362513 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:07.362586 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:43395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 14:39:07.362778 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 24434
W20260808 14:39:07.364939 24221 catalog_manager.cc:4584] TS deea4595b35b4e39ba640478ca997e3a (127.19.144.65:43395): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f4153efd72541c78d3cf8afae21c85f: Network error: Client connection negotiation failed: client connection to 127.19.144.65:43395: connect: Connection refused (error 111)
I20260808 14:39:07.371295 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 24574
I20260808 14:39:07.380764 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 24201
2026-08-08T14:39:07Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2471 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T14:39:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:07Z Disabled control of system clock
I20260808 14:39:07.415731 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:35799
--webserver_interface=127.19.144.126
--webserver_port=0
--builtin_ntp_servers=127.19.144.84:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:35799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 14:39:07.534024 24756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:07.534343 24756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:07.534382 24756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:07.538487 24756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:07.538563 24756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:07.538579 24756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:07.538596 24756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:07.538612 24756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:39:07.543596 24756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:35799
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:35799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:07.544775 24756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:07.546240 24756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:07.555485 24763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:07.555490 24762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:07.555473 24765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:07.557013 24756 server_base.cc:1034] running on GCE node
I20260808 14:39:07.557896 24756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:07.559572 24756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:07.561725 24756 hybrid_clock.cc:648] HybridClock initialized: now 1786199947561635 us; error 104 us; skew 500 ppm
I20260808 14:39:07.564724 24756 webserver.cc:460] Webserver started at http://127.19.144.126:45457/ using document root <none> and password file <none>
I20260808 14:39:07.565584 24756 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:07.565781 24756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:07.566121 24756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:07.569515 24756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "e2bf78e29a6c4cbd9a5e046aa68b5aac"
format_stamp: "Formatted at 2026-08-08 14:39:07 on dist-test-slave-p3t5"
server_key: "e83354ea1f60f65181f06bc370b449c2"
server_key_iv: "35b946660a31dea902e4df407ff4d542"
server_key_version: "encryptionkey@0"
I20260808 14:39:07.570408 24756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "e2bf78e29a6c4cbd9a5e046aa68b5aac"
format_stamp: "Formatted at 2026-08-08 14:39:07 on dist-test-slave-p3t5"
server_key: "e83354ea1f60f65181f06bc370b449c2"
server_key_iv: "35b946660a31dea902e4df407ff4d542"
server_key_version: "encryptionkey@0"
I20260808 14:39:07.576323 24756 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 14:39:07.580098 24771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:07.582324 24756 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:39:07.582505 24756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "e2bf78e29a6c4cbd9a5e046aa68b5aac"
format_stamp: "Formatted at 2026-08-08 14:39:07 on dist-test-slave-p3t5"
server_key: "e83354ea1f60f65181f06bc370b449c2"
server_key_iv: "35b946660a31dea902e4df407ff4d542"
server_key_version: "encryptionkey@0"
I20260808 14:39:07.582674 24756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:07.589557 24756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:07.590302 24756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:07.590471 24756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:07.597735 24756 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:35799
I20260808 14:39:07.597855 24824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:35799 every 8 connection(s)
I20260808 14:39:07.598860 24756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:07.602679 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 24756
I20260808 14:39:07.602780 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:39:07.602799 24825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:07.603109 20033 external_mini_cluster.cc:1428] Setting key c2197ec0354adc7babda41e95a9e63e8
I20260808 14:39:07.609570 24828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.604766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49316 (local address 127.19.144.126:35799)
0808 14:39:07.605554 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:07.605570 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:07.605609 (+ 39us) server_negotiation.cc:407] Connection header received
0808 14:39:07.606535 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:07.606557 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:07.606955 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:07.607325 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:07.608223 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:07.608228 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:07.608242 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:07.608254 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:07.608315 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:07.608331 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:07.608498 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:07.608576 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 14:39:07.608835 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":155,"threads_started":1}
I20260808 14:39:07.610721 24825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac: Bootstrap starting.
I20260808 14:39:07.613430 24825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:07.614306 24825 log.cc:824] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:07.616705 24825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac: No bootstrap required, opened a new log
I20260808 14:39:07.620741 24825 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bf78e29a6c4cbd9a5e046aa68b5aac" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35799 } }
I20260808 14:39:07.621047 24825 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:07.621145 24825 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2bf78e29a6c4cbd9a5e046aa68b5aac, State: Initialized, Role: FOLLOWER
I20260808 14:39:07.621731 24825 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bf78e29a6c4cbd9a5e046aa68b5aac" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35799 } }
I20260808 14:39:07.621910 24825 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:07.621984 24825 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:07.622094 24825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:07.623173 24825 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bf78e29a6c4cbd9a5e046aa68b5aac" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35799 } }
I20260808 14:39:07.623594 24825 leader_election.cc:302] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2bf78e29a6c4cbd9a5e046aa68b5aac; no voters:
I20260808 14:39:07.624002 24825 leader_election.cc:288] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:39:07.624104 24830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:07.624346 24830 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [term 1 LEADER]: Becoming Leader. State: Replica: e2bf78e29a6c4cbd9a5e046aa68b5aac, State: Running, Role: LEADER
I20260808 14:39:07.624663 24830 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bf78e29a6c4cbd9a5e046aa68b5aac" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35799 } }
I20260808 14:39:07.625519 24825 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:07.626600 24832 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2bf78e29a6c4cbd9a5e046aa68b5aac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bf78e29a6c4cbd9a5e046aa68b5aac" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35799 } } }
I20260808 14:39:07.626632 24831 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2bf78e29a6c4cbd9a5e046aa68b5aac. Latest consensus state: current_term: 1 leader_uuid: "e2bf78e29a6c4cbd9a5e046aa68b5aac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bf78e29a6c4cbd9a5e046aa68b5aac" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35799 } } }
I20260808 14:39:07.626871 24831 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:07.627177 24832 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:07.627310 24840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:07.630821 24840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:07.636837 24840 catalog_manager.cc:1347] Generated new cluster ID: 89f25cd2352c492793b5b626de7aaf4f
I20260808 14:39:07.636938 24840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:07.657341 24840 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:39:07.657796 24840 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:07.665154 24840 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac: Generated new TSK 0
I20260808 14:39:07.665998 24840 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:39:07.677894 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35799
--builtin_ntp_servers=127.19.144.84:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 14:39:07.802119 24850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:07.802403 24850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:07.802480 24850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:07.806828 24850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:07.806906 24850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:07.806973 24850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:39:07.812044 24850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:07.813365 24850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:07.815079 24850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:07.825666 24855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:07.825680 24858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:07.825687 24856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:07.826229 24850 server_base.cc:1034] running on GCE node
I20260808 14:39:07.826756 24850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:07.827533 24850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:07.828740 24850 hybrid_clock.cc:648] HybridClock initialized: now 1786199947828723 us; error 31 us; skew 500 ppm
I20260808 14:39:07.830958 24850 webserver.cc:460] Webserver started at http://127.19.144.65:35271/ using document root <none> and password file <none>
I20260808 14:39:07.831606 24850 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:07.831705 24850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:07.831954 24850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:07.834193 24850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "6b4e907eae2846619d2c5f4821fcbfa5"
format_stamp: "Formatted at 2026-08-08 14:39:07 on dist-test-slave-p3t5"
server_key: "94bf4dbf03cbdb9db5cc21f161b818ad"
server_key_iv: "85dcded89f160a56a6941622a0b00ed9"
server_key_version: "encryptionkey@0"
I20260808 14:39:07.834791 24850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "6b4e907eae2846619d2c5f4821fcbfa5"
format_stamp: "Formatted at 2026-08-08 14:39:07 on dist-test-slave-p3t5"
server_key: "94bf4dbf03cbdb9db5cc21f161b818ad"
server_key_iv: "85dcded89f160a56a6941622a0b00ed9"
server_key_version: "encryptionkey@0"
I20260808 14:39:07.838658 24850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:39:07.841464 24864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:07.842968 24850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 14:39:07.843123 24850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "6b4e907eae2846619d2c5f4821fcbfa5"
format_stamp: "Formatted at 2026-08-08 14:39:07 on dist-test-slave-p3t5"
server_key: "94bf4dbf03cbdb9db5cc21f161b818ad"
server_key_iv: "85dcded89f160a56a6941622a0b00ed9"
server_key_version: "encryptionkey@0"
I20260808 14:39:07.843255 24850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:07.848413 24850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:07.849068 24850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:07.849267 24850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:07.849918 24850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:07.850948 24850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:07.851025 24850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:07.851104 24850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:07.851155 24850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:07.860570 24850 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:35973
I20260808 14:39:07.860590 24983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:35973 every 8 connection(s)
I20260808 14:39:07.861595 24850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:39:07.866034 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 24850
I20260808 14:39:07.866142 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:39:07.866426 20033 external_mini_cluster.cc:1428] Setting key be95679529e1f1b79fe60bdb4b923287
I20260808 14:39:07.868450 24828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.863687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:36283 (local address 127.19.144.126:35799)
0808 14:39:07.863846 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:07.863850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:07.865007 (+ 1157us) server_negotiation.cc:407] Connection header received
0808 14:39:07.866592 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:07.866596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:07.866643 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:07.866779 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:07.867969 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:07.867973 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:07.867976 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:07.867988 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:07.868016 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:07.868024 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:07.868118 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:07.868273 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 14:39:07.868319 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:39:07.868549 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35799
--builtin_ntp_servers=127.19.144.84:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 14:39:07.869441 24987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:07.864117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35799 (local address 127.19.144.65:36283)
0808 14:39:07.864843 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:07.864888 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:07.866344 (+ 1456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:07.866953 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:07.866960 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:07.867053 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:07.867677 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:39:07.867679 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:07.867742 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:39:07.867819 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:07.868107 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:07.868116 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:07.868123 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 14:39:07.868323 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 14:39:07.868579 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":127,"threads_started":1}
I20260808 14:39:07.871321 24984 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35799
I20260808 14:39:07.871582 24984 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:07.872133 24984 heartbeater.cc:511] Master 127.19.144.126:35799 requested a full tablet report, sending...
I20260808 14:39:07.873942 24788 ts_manager.cc:194] Registered new tserver with Master: 6b4e907eae2846619d2c5f4821fcbfa5 (127.19.144.65:35973)
W20260808 14:39:07.990855 24988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:07.991211 24988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:07.991290 24988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:07.995476 24988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:07.995589 24988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:07.995687 24988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:39:08.000801 24988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:08.002172 24988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:08.003690 24988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:08.014374 24994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:08.014372 24993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:08.014442 24988 server_base.cc:1034] running on GCE node
W20260808 14:39:08.014376 24996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:08.015017 24988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:08.015759 24988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:08.016958 24988 hybrid_clock.cc:648] HybridClock initialized: now 1786199948016941 us; error 28 us; skew 500 ppm
I20260808 14:39:08.019145 24988 webserver.cc:460] Webserver started at http://127.19.144.66:37571/ using document root <none> and password file <none>
I20260808 14:39:08.019776 24988 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:08.019845 24988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:08.020074 24988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:08.022389 24988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "72ac1bd6fce54e0781abd3d051a0a0b3"
format_stamp: "Formatted at 2026-08-08 14:39:08 on dist-test-slave-p3t5"
server_key: "777fab1160e79cee10ef2a3037e1368e"
server_key_iv: "5c53f6e49b640ee16dfdfc3e0c2570b3"
server_key_version: "encryptionkey@0"
I20260808 14:39:08.023001 24988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "72ac1bd6fce54e0781abd3d051a0a0b3"
format_stamp: "Formatted at 2026-08-08 14:39:08 on dist-test-slave-p3t5"
server_key: "777fab1160e79cee10ef2a3037e1368e"
server_key_iv: "5c53f6e49b640ee16dfdfc3e0c2570b3"
server_key_version: "encryptionkey@0"
I20260808 14:39:08.027288 24988 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:39:08.030277 25002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:08.031664 24988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:39:08.031811 24988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "72ac1bd6fce54e0781abd3d051a0a0b3"
format_stamp: "Formatted at 2026-08-08 14:39:08 on dist-test-slave-p3t5"
server_key: "777fab1160e79cee10ef2a3037e1368e"
server_key_iv: "5c53f6e49b640ee16dfdfc3e0c2570b3"
server_key_version: "encryptionkey@0"
I20260808 14:39:08.031950 24988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:08.037065 24988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:08.038041 24988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:08.038267 24988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:08.038906 24988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:08.039893 24988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:08.039943 24988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:08.040014 24988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:08.040055 24988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:08.049414 24988 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:46637
I20260808 14:39:08.049469 25119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:46637 every 8 connection(s)
I20260808 14:39:08.050469 24988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:39:08.057195 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 24988
I20260808 14:39:08.057204 24828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.052550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:59177 (local address 127.19.144.126:35799)
0808 14:39:08.052693 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:08.052697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:08.053926 (+ 1229us) server_negotiation.cc:407] Connection header received
0808 14:39:08.055322 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:08.055326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:08.055375 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:08.055460 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:08.056643 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:08.056646 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:08.056649 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:08.056668 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:08.056690 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:08.056698 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:08.056850 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:08.057002 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 14:39:08.057055 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 14:39:08.057376 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:39:08.057798 20033 external_mini_cluster.cc:1428] Setting key 5d55813b4acdb6c43ac5001a1dcb1ca4
I20260808 14:39:08.058295 25122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.052951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35799 (local address 127.19.144.66:59177)
0808 14:39:08.053754 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:08.053806 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:08.055081 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:08.055659 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:08.055666 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:08.055764 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:08.056363 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:39:08.056365 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:08.056408 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:39:08.056482 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:08.056829 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:08.056838 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:08.056844 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 14:39:08.057041 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 14:39:08.057309 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":127,"threads_started":1}
I20260808 14:39:08.059774 25120 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35799
I20260808 14:39:08.060043 25120 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:08.060323 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35799
--builtin_ntp_servers=127.19.144.84:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 14:39:08.060549 25120 heartbeater.cc:511] Master 127.19.144.126:35799 requested a full tablet report, sending...
I20260808 14:39:08.061853 24788 ts_manager.cc:194] Registered new tserver with Master: 72ac1bd6fce54e0781abd3d051a0a0b3 (127.19.144.66:46637)
W20260808 14:39:08.183202 25123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:08.183525 25123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:08.183568 25123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:08.187863 25123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:08.187959 25123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:08.188050 25123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:39:08.193183 25123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:08.194469 25123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:08.195955 25123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:08.206178 25131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:08.206178 25133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:08.206166 25130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:08.206632 25123 server_base.cc:1034] running on GCE node
I20260808 14:39:08.207034 25123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:08.207713 25123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:08.208966 25123 hybrid_clock.cc:648] HybridClock initialized: now 1786199948208947 us; error 33 us; skew 500 ppm
I20260808 14:39:08.211238 25123 webserver.cc:460] Webserver started at http://127.19.144.67:41355/ using document root <none> and password file <none>
I20260808 14:39:08.211900 25123 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:08.211997 25123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:08.212219 25123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:08.214464 25123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "2bba6bb6fa474639b0ba84d94032c48a"
format_stamp: "Formatted at 2026-08-08 14:39:08 on dist-test-slave-p3t5"
server_key: "716dd01f95789bf35ab9fe0ecf176d98"
server_key_iv: "76146ef852f23e9bda5c03750d48cb16"
server_key_version: "encryptionkey@0"
I20260808 14:39:08.215092 25123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "2bba6bb6fa474639b0ba84d94032c48a"
format_stamp: "Formatted at 2026-08-08 14:39:08 on dist-test-slave-p3t5"
server_key: "716dd01f95789bf35ab9fe0ecf176d98"
server_key_iv: "76146ef852f23e9bda5c03750d48cb16"
server_key_version: "encryptionkey@0"
I20260808 14:39:08.219252 25123 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:39:08.222214 25139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:08.223620 25123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:08.223775 25123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "2bba6bb6fa474639b0ba84d94032c48a"
format_stamp: "Formatted at 2026-08-08 14:39:08 on dist-test-slave-p3t5"
server_key: "716dd01f95789bf35ab9fe0ecf176d98"
server_key_iv: "76146ef852f23e9bda5c03750d48cb16"
server_key_version: "encryptionkey@0"
I20260808 14:39:08.223913 25123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:08.229197 25123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:08.229984 25123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:08.230188 25123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:08.230772 25123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:08.231786 25123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:08.231858 25123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:08.231937 25123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:08.231982 25123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:08.241295 25123 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:39509
I20260808 14:39:08.241314 25255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:39509 every 8 connection(s)
I20260808 14:39:08.242347 25123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:39:08.249051 24828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.244321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:38641 (local address 127.19.144.126:35799)
0808 14:39:08.244482 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:08.244487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:08.245847 (+ 1360us) server_negotiation.cc:407] Connection header received
0808 14:39:08.247229 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:08.247232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:08.247313 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:08.247427 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:08.248574 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:08.248577 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:08.248579 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:08.248591 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:08.248610 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:08.248617 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:08.248697 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:08.248808 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 14:39:08.248867 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:39:08.249167 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 25123
I20260808 14:39:08.249276 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:39:08.249559 20033 external_mini_cluster.cc:1428] Setting key 5b47fa35bf52b1d97093d424e53d47b2
I20260808 14:39:08.250119 25258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.244800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35799 (local address 127.19.144.67:38641)
0808 14:39:08.245699 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:08.245743 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:08.247006 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:08.247581 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:08.247588 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:08.247681 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:08.248238 (+ 557us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:39:08.248240 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:08.248284 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:39:08.248384 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:08.248682 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:08.248686 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:08.248690 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 14:39:08.248831 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 14:39:08.249044 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":116,"threads_started":1}
I20260808 14:39:08.251436 25256 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35799
I20260808 14:39:08.251732 25256 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:08.252250 25256 heartbeater.cc:511] Master 127.19.144.126:35799 requested a full tablet report, sending...
I20260808 14:39:08.253244 24788 ts_manager.cc:194] Registered new tserver with Master: 2bba6bb6fa474639b0ba84d94032c48a (127.19.144.67:39509)
I20260808 14:39:08.264132 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:39:08.267876 24828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.266325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49326 (local address 127.19.144.126:35799)
0808 14:39:08.266476 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:08.266479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:08.266609 (+ 130us) server_negotiation.cc:407] Connection header received
0808 14:39:08.266677 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:08.266679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:08.266728 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:08.266821 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:08.267530 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:08.267533 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:08.267535 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:08.267546 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:08.267568 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:08.267575 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:08.267646 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:08.267712 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 14:39:08.267748 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 14:39:08.271091 24788 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:39:08.274331 24788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:39:08.287009 25269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.283610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:46637 (local address 127.0.0.1:60058)
0808 14:39:08.284039 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:08.284091 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:08.284199 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:08.285181 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:08.285188 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:08.285273 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:08.285990 (+ 717us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:39:08.285994 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:08.286022 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:39:08.286103 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:08.286580 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:08.286589 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:08.286596 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 14:39:08.286686 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 14:39:08.286816 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":118,"threads_started":1}
I20260808 14:39:08.287633 25274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.283849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:35973 (local address 127.0.0.1:40666)
0808 14:39:08.285000 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:08.285015 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:08.285108 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:08.286357 (+ 1249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:08.286360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:08.286377 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:08.286829 (+ 452us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:39:08.286831 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:08.286841 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:39:08.286868 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:08.287372 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:08.287378 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:08.287382 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 14:39:08.287474 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 14:39:08.287510 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1086,"thread_start_us":108,"threads_started":1}
I20260808 14:39:08.287722 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.283766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60058 (local address 127.19.144.66:46637)
0808 14:39:08.284624 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:08.284637 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:08.284667 (+ 30us) server_negotiation.cc:407] Connection header received
0808 14:39:08.284730 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:08.284735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:08.285000 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:08.285597 (+ 597us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:08.286350 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:08.286356 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:08.286369 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:08.286379 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:08.286424 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:08.286455 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:08.287435 (+ 980us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:08.287488 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 14:39:08.287571 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":137,"threads_started":1}
I20260808 14:39:08.288319 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.284337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40666 (local address 127.19.144.65:35973)
0808 14:39:08.284829 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:08.284840 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:08.285206 (+ 366us) server_negotiation.cc:407] Connection header received
0808 14:39:08.285261 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:08.285267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:08.285467 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:08.285596 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:08.286996 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:08.287003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:08.287015 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:08.287030 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:08.287078 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:08.287119 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:08.287244 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:08.287592 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 14:39:08.287676 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":135,"threads_started":1}
I20260808 14:39:08.288545 25271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.284441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:39509 (local address 127.0.0.1:56452)
0808 14:39:08.285384 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:08.285397 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:08.285926 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:08.286411 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:08.286414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:08.286429 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:08.287780 (+ 1351us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:39:08.287783 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:08.287795 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:39:08.287821 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:08.288299 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:08.288302 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:08.288306 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 14:39:08.288393 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 14:39:08.288435 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"thread_start_us":106,"threads_started":1}
I20260808 14:39:08.288949 25275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.284619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56452 (local address 127.19.144.67:39509)
0808 14:39:08.285095 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:08.285110 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:08.285504 (+ 394us) server_negotiation.cc:407] Connection header received
0808 14:39:08.286083 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:08.286089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:08.286280 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:08.286426 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:08.287958 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:08.287966 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:08.287980 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:08.287995 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:08.288111 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:08.288150 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:08.288253 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:08.288593 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 14:39:08.288794 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":167,"threads_started":1}
I20260808 14:39:08.291092 25052 tablet_service.cc:1467] Processing CreateTablet for tablet 2366738adc3f4939a3b81d73e04c5840 (DEFAULT_TABLE table=test-table [id=12be75dedb924970b580fa2d2277aef4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:08.291086 24917 tablet_service.cc:1467] Processing CreateTablet for tablet 2366738adc3f4939a3b81d73e04c5840 (DEFAULT_TABLE table=test-table [id=12be75dedb924970b580fa2d2277aef4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:08.292224 25052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2366738adc3f4939a3b81d73e04c5840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:08.292258 24917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2366738adc3f4939a3b81d73e04c5840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:08.292392 25189 tablet_service.cc:1467] Processing CreateTablet for tablet 2366738adc3f4939a3b81d73e04c5840 (DEFAULT_TABLE table=test-table [id=12be75dedb924970b580fa2d2277aef4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:08.293411 25189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2366738adc3f4939a3b81d73e04c5840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:08.298794 25276 tablet_bootstrap.cc:492] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3: Bootstrap starting.
I20260808 14:39:08.300501 25278 tablet_bootstrap.cc:492] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a: Bootstrap starting.
I20260808 14:39:08.300802 25277 tablet_bootstrap.cc:492] T 2366738adc3f4939a3b81d73e04c5840 P 6b4e907eae2846619d2c5f4821fcbfa5: Bootstrap starting.
I20260808 14:39:08.302227 25276 tablet_bootstrap.cc:654] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:08.302656 25278 tablet_bootstrap.cc:654] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:08.303022 25277 tablet_bootstrap.cc:654] T 2366738adc3f4939a3b81d73e04c5840 P 6b4e907eae2846619d2c5f4821fcbfa5: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:08.303182 25276 log.cc:824] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:08.303467 25278 log.cc:824] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:08.303825 25277 log.cc:824] T 2366738adc3f4939a3b81d73e04c5840 P 6b4e907eae2846619d2c5f4821fcbfa5: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:08.305714 25278 tablet_bootstrap.cc:492] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a: No bootstrap required, opened a new log
I20260808 14:39:08.305783 25276 tablet_bootstrap.cc:492] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3: No bootstrap required, opened a new log
I20260808 14:39:08.305949 25278 ts_tablet_manager.cc:1397] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 14:39:08.305987 25276 ts_tablet_manager.cc:1397] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 14:39:08.306663 25277 tablet_bootstrap.cc:492] T 2366738adc3f4939a3b81d73e04c5840 P 6b4e907eae2846619d2c5f4821fcbfa5: No bootstrap required, opened a new log
I20260808 14:39:08.306870 25277 ts_tablet_manager.cc:1397] T 2366738adc3f4939a3b81d73e04c5840 P 6b4e907eae2846619d2c5f4821fcbfa5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 14:39:08.309767 25276 raft_consensus.cc:348] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ac1bd6fce54e0781abd3d051a0a0b3" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46637 } } peers { permanent_uuid: "2bba6bb6fa474639b0ba84d94032c48a" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39509 } } peers { permanent_uuid: "6b4e907eae2846619d2c5f4821fcbfa5" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 35973 } }
I20260808 14:39:08.310037 25276 raft_consensus.cc:374] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:08.310096 25276 raft_consensus.cc:729] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ac1bd6fce54e0781abd3d051a0a0b3, State: Initialized, Role: FOLLOWER
I20260808 14:39:08.310599 25276 consensus_queue.cc:260] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ac1bd6fce54e0781abd3d051a0a0b3" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46637 } } peers { permanent_uuid: "2bba6bb6fa474639b0ba84d94032c48a" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39509 } } peers { permanent_uuid: "6b4e907eae2846619d2c5f4821fcbfa5" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 35973 } }
I20260808 14:39:08.310750 25278 raft_consensus.cc:348] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ac1bd6fce54e0781abd3d051a0a0b3" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46637 } } peers { permanent_uuid: "2bba6bb6fa474639b0ba84d94032c48a" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39509 } } peers { permanent_uuid: "6b4e907eae2846619d2c5f4821fcbfa5" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 35973 } }
I20260808 14:39:08.311034 25278 raft_consensus.cc:374] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:08.311100 25278 raft_consensus.cc:729] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bba6bb6fa474639b0ba84d94032c48a, State: Initialized, Role: FOLLOWER
I20260808 14:39:08.311494 25120 heartbeater.cc:503] Master 127.19.144.126:35799 was elected leader, sending a full tablet report...
I20260808 14:39:08.311367 25277 raft_consensus.cc:348] T 2366738adc3f4939a3b81d73e04c5840 P 6b4e907eae2846619d2c5f4821fcbfa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ac1bd6fce54e0781abd3d051a0a0b3" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46637 } } peers { permanent_uuid: "2bba6bb6fa474639b0ba84d94032c48a" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39509 } } peers { permanent_uuid: "6b4e907eae2846619d2c5f4821fcbfa5" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 35973 } }
I20260808 14:39:08.311606 25277 raft_consensus.cc:374] T 2366738adc3f4939a3b81d73e04c5840 P 6b4e907eae2846619d2c5f4821fcbfa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:08.311533 25278 consensus_queue.cc:260] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ac1bd6fce54e0781abd3d051a0a0b3" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46637 } } peers { permanent_uuid: "2bba6bb6fa474639b0ba84d94032c48a" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39509 } } peers { permanent_uuid: "6b4e907eae2846619d2c5f4821fcbfa5" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 35973 } }
I20260808 14:39:08.311664 25277 raft_consensus.cc:729] T 2366738adc3f4939a3b81d73e04c5840 P 6b4e907eae2846619d2c5f4821fcbfa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b4e907eae2846619d2c5f4821fcbfa5, State: Initialized, Role: FOLLOWER
I20260808 14:39:08.312117 25277 consensus_queue.cc:260] T 2366738adc3f4939a3b81d73e04c5840 P 6b4e907eae2846619d2c5f4821fcbfa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ac1bd6fce54e0781abd3d051a0a0b3" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46637 } } peers { permanent_uuid: "2bba6bb6fa474639b0ba84d94032c48a" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39509 } } peers { permanent_uuid: "6b4e907eae2846619d2c5f4821fcbfa5" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 35973 } }
I20260808 14:39:08.312618 25278 ts_tablet_manager.cc:1428] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 14:39:08.312705 25256 heartbeater.cc:503] Master 127.19.144.126:35799 was elected leader, sending a full tablet report...
I20260808 14:39:08.313090 24984 heartbeater.cc:503] Master 127.19.144.126:35799 was elected leader, sending a full tablet report...
I20260808 14:39:08.313076 25277 ts_tablet_manager.cc:1428] T 2366738adc3f4939a3b81d73e04c5840 P 6b4e907eae2846619d2c5f4821fcbfa5: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260808 14:39:08.314965 25276 ts_tablet_manager.cc:1428] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
W20260808 14:39:08.362977 24985 tablet.cc:2369] T 2366738adc3f4939a3b81d73e04c5840 P 6b4e907eae2846619d2c5f4821fcbfa5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:39:08.418296 25283 raft_consensus.cc:482] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:39:08.418571 25283 raft_consensus.cc:504] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ac1bd6fce54e0781abd3d051a0a0b3" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46637 } } peers { permanent_uuid: "2bba6bb6fa474639b0ba84d94032c48a" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39509 } } peers { permanent_uuid: "6b4e907eae2846619d2c5f4821fcbfa5" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 35973 } }
I20260808 14:39:08.419962 25283 leader_election.cc:288] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72ac1bd6fce54e0781abd3d051a0a0b3 (127.19.144.66:46637), 6b4e907eae2846619d2c5f4821fcbfa5 (127.19.144.65:35973)
I20260808 14:39:08.422240 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.420430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:36841 (local address 127.19.144.66:46637)
0808 14:39:08.420565 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:08.420572 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:08.420591 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:39:08.420642 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:08.420645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:08.420684 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:08.420769 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:08.421725 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:08.421729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:08.421731 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:08.421743 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:08.421764 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:08.421773 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:08.421929 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:08.422044 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 14:39:08.422095 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:39:08.422230 25258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.420196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:46637 (local address 127.19.144.67:36841)
0808 14:39:08.420386 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:08.420403 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:08.420532 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:08.420808 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:08.420810 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:08.420824 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:08.421523 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:39:08.421525 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:08.421535 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:39:08.421561 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:08.421934 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:08.421936 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:08.421939 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 14:39:08.422041 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 14:39:08.422084 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260808 14:39:08.422263 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.420473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:33689 (local address 127.19.144.65:35973)
0808 14:39:08.420612 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:08.420616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:08.420889 (+ 273us) server_negotiation.cc:407] Connection header received
0808 14:39:08.420950 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:08.420953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:08.421000 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:08.421098 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:08.421887 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:08.421893 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:08.421896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:08.421909 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:08.421930 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:08.421942 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:08.422031 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:08.422117 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 14:39:08.422155 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:39:08.422243 25285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.420353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:35973 (local address 127.19.144.67:33689)
0808 14:39:08.420779 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:08.420793 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:08.420884 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:08.421175 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:08.421179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:08.421198 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:08.421720 (+ 522us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:39:08.421722 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:08.421733 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:39:08.421758 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:08.422028 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:08.422031 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:08.422034 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 14:39:08.422106 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 14:39:08.422132 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":160,"threads_started":1}
I20260808 14:39:08.423097 25073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2366738adc3f4939a3b81d73e04c5840" candidate_uuid: "2bba6bb6fa474639b0ba84d94032c48a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ac1bd6fce54e0781abd3d051a0a0b3" is_pre_election: true
I20260808 14:39:08.423111 24937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2366738adc3f4939a3b81d73e04c5840" candidate_uuid: "2bba6bb6fa474639b0ba84d94032c48a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b4e907eae2846619d2c5f4821fcbfa5" is_pre_election: true
I20260808 14:39:08.423429 25073 raft_consensus.cc:2454] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bba6bb6fa474639b0ba84d94032c48a in term 0.
I20260808 14:39:08.423442 24937 raft_consensus.cc:2454] T 2366738adc3f4939a3b81d73e04c5840 P 6b4e907eae2846619d2c5f4821fcbfa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bba6bb6fa474639b0ba84d94032c48a in term 0.
I20260808 14:39:08.423980 25143 leader_election.cc:302] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2bba6bb6fa474639b0ba84d94032c48a, 6b4e907eae2846619d2c5f4821fcbfa5, 72ac1bd6fce54e0781abd3d051a0a0b3; no voters:
I20260808 14:39:08.424389 25283 raft_consensus.cc:2781] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:39:08.424484 25283 raft_consensus.cc:482] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:39:08.424516 25283 raft_consensus.cc:3037] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:08.425679 25283 raft_consensus.cc:504] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ac1bd6fce54e0781abd3d051a0a0b3" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46637 } } peers { permanent_uuid: "2bba6bb6fa474639b0ba84d94032c48a" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39509 } } peers { permanent_uuid: "6b4e907eae2846619d2c5f4821fcbfa5" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 35973 } }
I20260808 14:39:08.426069 25283 leader_election.cc:288] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [CANDIDATE]: Term 1 election: Requested vote from peers 72ac1bd6fce54e0781abd3d051a0a0b3 (127.19.144.66:46637), 6b4e907eae2846619d2c5f4821fcbfa5 (127.19.144.65:35973)
I20260808 14:39:08.426510 25073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2366738adc3f4939a3b81d73e04c5840" candidate_uuid: "2bba6bb6fa474639b0ba84d94032c48a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ac1bd6fce54e0781abd3d051a0a0b3"
I20260808 14:39:08.426677 25073 raft_consensus.cc:3037] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:08.427940 25073 raft_consensus.cc:2454] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bba6bb6fa474639b0ba84d94032c48a in term 1.
I20260808 14:39:08.427968 24937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2366738adc3f4939a3b81d73e04c5840" candidate_uuid: "2bba6bb6fa474639b0ba84d94032c48a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b4e907eae2846619d2c5f4821fcbfa5"
I20260808 14:39:08.428179 24937 raft_consensus.cc:3037] T 2366738adc3f4939a3b81d73e04c5840 P 6b4e907eae2846619d2c5f4821fcbfa5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:08.428297 25143 leader_election.cc:302] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bba6bb6fa474639b0ba84d94032c48a, 72ac1bd6fce54e0781abd3d051a0a0b3; no voters:
I20260808 14:39:08.428488 25283 raft_consensus.cc:2781] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:08.428755 25283 raft_consensus.cc:686] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [term 1 LEADER]: Becoming Leader. State: Replica: 2bba6bb6fa474639b0ba84d94032c48a, State: Running, Role: LEADER
I20260808 14:39:08.429127 25283 consensus_queue.cc:237] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ac1bd6fce54e0781abd3d051a0a0b3" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46637 } } peers { permanent_uuid: "2bba6bb6fa474639b0ba84d94032c48a" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39509 } } peers { permanent_uuid: "6b4e907eae2846619d2c5f4821fcbfa5" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 35973 } }
I20260808 14:39:08.429538 24937 raft_consensus.cc:2454] T 2366738adc3f4939a3b81d73e04c5840 P 6b4e907eae2846619d2c5f4821fcbfa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bba6bb6fa474639b0ba84d94032c48a in term 1.
I20260808 14:39:08.432953 24787 catalog_manager.cc:5495] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2bba6bb6fa474639b0ba84d94032c48a (127.19.144.67). New cstate: current_term: 1 leader_uuid: "2bba6bb6fa474639b0ba84d94032c48a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ac1bd6fce54e0781abd3d051a0a0b3" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bba6bb6fa474639b0ba84d94032c48a" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b4e907eae2846619d2c5f4821fcbfa5" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 35973 } health_report { overall_health: UNKNOWN } } }
I20260808 14:39:08.443370 25275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.441539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.19.144.67:39509)
0808 14:39:08.441734 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:08.441739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:08.441754 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:39:08.441882 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:08.441885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:08.441955 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:08.442055 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:08.442881 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:08.442885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:08.442888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:08.442907 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:08.442928 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:08.442937 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:08.443040 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:08.443152 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 14:39:08.443197 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 14:39:08.450574 24937 raft_consensus.cc:1261] T 2366738adc3f4939a3b81d73e04c5840 P 6b4e907eae2846619d2c5f4821fcbfa5 [term 1 FOLLOWER]: Refusing update from remote peer 2bba6bb6fa474639b0ba84d94032c48a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:08.450574 25073 raft_consensus.cc:1261] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3 [term 1 FOLLOWER]: Refusing update from remote peer 2bba6bb6fa474639b0ba84d94032c48a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:08.451433 25286 consensus_queue.cc:1035] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b4e907eae2846619d2c5f4821fcbfa5" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 35973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:08.451661 25283 consensus_queue.cc:1035] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [LEADER]: Connected to new peer: Peer: permanent_uuid: "72ac1bd6fce54e0781abd3d051a0a0b3" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:08.459914 25289 mvcc.cc:205] Tried to move back new op lower bound from 7316274988845199360 to 7316274988769374208. Current Snapshot: MvccSnapshot[applied={T|T < 7316274988845199360}]
I20260808 14:39:08.461269 25290 mvcc.cc:205] Tried to move back new op lower bound from 7316274988845199360 to 7316274988769374208. Current Snapshot: MvccSnapshot[applied={T|T < 7316274988845199360}]
I20260808 14:39:08.461274 25291 mvcc.cc:205] Tried to move back new op lower bound from 7316274988845199360 to 7316274988769374208. Current Snapshot: MvccSnapshot[applied={T|T < 7316274988845199360}]
I20260808 14:39:08.468838 24787 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:08.469043 24787 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:08.471653 24787 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2bf78e29a6c4cbd9a5e046aa68b5aac: Sending DeleteTablet for 3 replicas of tablet 2366738adc3f4939a3b81d73e04c5840
I20260808 14:39:08.472661 24917 tablet_service.cc:1514] Processing DeleteTablet for tablet 2366738adc3f4939a3b81d73e04c5840 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:08 UTC) from {username='slave'} at 127.0.0.1:40666
I20260808 14:39:08.472661 25189 tablet_service.cc:1514] Processing DeleteTablet for tablet 2366738adc3f4939a3b81d73e04c5840 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:08 UTC) from {username='slave'} at 127.0.0.1:56452
I20260808 14:39:08.472730 25052 tablet_service.cc:1514] Processing DeleteTablet for tablet 2366738adc3f4939a3b81d73e04c5840 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:08 UTC) from {username='slave'} at 127.0.0.1:60058
I20260808 14:39:08.473476 25300 tablet_replica.cc:330] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3: stopping tablet replica
I20260808 14:39:08.473934 25300 raft_consensus.cc:2229] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:08.474067 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 24850
I20260808 14:39:08.474313 25300 raft_consensus.cc:2258] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:08.475252 25299 tablet_replica.cc:330] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a: stopping tablet replica
I20260808 14:39:08.475607 25299 raft_consensus.cc:2229] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:39:08.476096 25299 raft_consensus.cc:2258] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:08.477839 25299 ts_tablet_manager.cc:1905] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:08.478259 25300 ts_tablet_manager.cc:1905] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:08.480584 25300 ts_tablet_manager.cc:1918] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:08.480718 25300 log.cc:1196] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/2366738adc3f4939a3b81d73e04c5840
I20260808 14:39:08.481071 25300 ts_tablet_manager.cc:1939] T 2366738adc3f4939a3b81d73e04c5840 P 72ac1bd6fce54e0781abd3d051a0a0b3: Deleting consensus metadata
I20260808 14:39:08.481643 25299 ts_tablet_manager.cc:1918] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:08.481779 25299 log.cc:1196] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/2366738adc3f4939a3b81d73e04c5840
I20260808 14:39:08.482137 25299 ts_tablet_manager.cc:1939] T 2366738adc3f4939a3b81d73e04c5840 P 2bba6bb6fa474639b0ba84d94032c48a: Deleting consensus metadata
I20260808 14:39:08.482646 24775 catalog_manager.cc:4841] TS 72ac1bd6fce54e0781abd3d051a0a0b3 (127.19.144.66:46637): tablet 2366738adc3f4939a3b81d73e04c5840 (table test-table [id=12be75dedb924970b580fa2d2277aef4]) successfully deleted
I20260808 14:39:08.483366 24775 catalog_manager.cc:4841] TS 2bba6bb6fa474639b0ba84d94032c48a (127.19.144.67:39509): tablet 2366738adc3f4939a3b81d73e04c5840 (table test-table [id=12be75dedb924970b580fa2d2277aef4]) successfully deleted
W20260808 14:39:08.484292 24772 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.144.65:35973 (error 108)
I20260808 14:39:08.484552 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 24988
I20260808 14:39:08.485941 25271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.485540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:35973 (local address 127.0.0.1:40672)
0808 14:39:08.485763 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:08.485844 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:35973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260808 14:39:08.486105 24772 catalog_manager.cc:4584] TS 6b4e907eae2846619d2c5f4821fcbfa5 (127.19.144.65:35973): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2366738adc3f4939a3b81d73e04c5840: Network error: Client connection negotiation failed: client connection to 127.19.144.65:35973: connect: Connection refused (error 111)
I20260808 14:39:08.493224 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 25123
I20260808 14:39:08.499780 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 24756
2026-08-08T14:39:08Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1117 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 14:39:08.523581 20033 mini_oidc.cc:152] Starting JWKS server
I20260808 14:39:08.525380 20033 webserver.cc:460] Webserver started at https://127.0.0.1:46593/ using document root <none> and password file <none>
I20260808 14:39:08.525585 20033 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 14:39:08.525938 20033 webserver.cc:460] Webserver started at http://127.0.0.1:36587/ using document root <none> and password file <none>
2026-08-08T14:39:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:08Z Disabled control of system clock
I20260808 14:39:08.537123 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:45963
--webserver_interface=127.19.144.126
--webserver_port=0
--builtin_ntp_servers=127.19.144.84:34329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:45963
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46593/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/testchainca.cert with env {}
W20260808 14:39:08.660758 25307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:08.661082 25307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:08.661124 25307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:08.665432 25307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:08.665508 25307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:08.665524 25307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:08.665541 25307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:08.665557 25307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 14:39:08.665573 25307 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 14:39:08.665659 25307 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46593/jwks.json
I20260808 14:39:08.671368 25307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:45963
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:45963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46593/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:08.672572 25307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:08.673765 25307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:39:08.683130 25307 server_base.cc:1034] running on GCE node
W20260808 14:39:08.683108 25315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:08.683115 25312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:08.683460 25313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:08.684111 25307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:08.685245 25307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:08.686419 25307 hybrid_clock.cc:648] HybridClock initialized: now 1786199948686383 us; error 46 us; skew 500 ppm
I20260808 14:39:08.688516 25307 webserver.cc:460] Webserver started at http://127.19.144.126:34981/ using document root <none> and password file <none>
I20260808 14:39:08.689129 25307 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:08.689195 25307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:08.689448 25307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:08.691648 25307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "8115ab0e73834c7885d899bb89f6b60d"
format_stamp: "Formatted at 2026-08-08 14:39:08 on dist-test-slave-p3t5"
server_key: "600b930493b1b3197d71f77f89d90837"
server_key_iv: "553fd9277c37eaf3afbc2f387365e3cd"
server_key_version: "encryptionkey@0"
I20260808 14:39:08.692242 25307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "8115ab0e73834c7885d899bb89f6b60d"
format_stamp: "Formatted at 2026-08-08 14:39:08 on dist-test-slave-p3t5"
server_key: "600b930493b1b3197d71f77f89d90837"
server_key_iv: "553fd9277c37eaf3afbc2f387365e3cd"
server_key_version: "encryptionkey@0"
I20260808 14:39:08.696138 25307 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 14:39:08.698792 25322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:08.700096 25307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:39:08.700249 25307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "8115ab0e73834c7885d899bb89f6b60d"
format_stamp: "Formatted at 2026-08-08 14:39:08 on dist-test-slave-p3t5"
server_key: "600b930493b1b3197d71f77f89d90837"
server_key_iv: "553fd9277c37eaf3afbc2f387365e3cd"
server_key_version: "encryptionkey@0"
I20260808 14:39:08.700390 25307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:08.728296 25307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:08.729110 25307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:08.729341 25307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:08.736955 25307 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:45963
I20260808 14:39:08.736964 25375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:45963 every 8 connection(s)
I20260808 14:39:08.738044 25307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:08.741405 25376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:08.744647 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 25307
I20260808 14:39:08.744779 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:39:08.745064 20033 external_mini_cluster.cc:1428] Setting key 4a21b92eb99b9933575bdd55a3f3221d
I20260808 14:39:08.748070 25376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d: Bootstrap starting.
I20260808 14:39:08.750805 25376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:08.751765 25376 log.cc:824] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:08.754472 25376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d: No bootstrap required, opened a new log
I20260808 14:39:08.755661 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.746668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40016 (local address 127.19.144.126:45963)
0808 14:39:08.747372 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:08.747390 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:08.747429 (+ 39us) server_negotiation.cc:407] Connection header received
0808 14:39:08.748101 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:08.748120 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:08.748605 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:08.748911 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:08.749308 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:08.750413 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:08.751701 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:08.752270 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:08.752401 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:08.753393 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:08.753400 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:08.753417 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:08.754032 (+ 615us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:08.754120 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:08.754149 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:08.754259 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:08.754364 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 14:39:08.754656 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":150,"threads_started":1}
I20260808 14:39:08.759016 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:39:08.759153 25376 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8115ab0e73834c7885d899bb89f6b60d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45963 } }
I20260808 14:39:08.759450 25376 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:08.759532 25376 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8115ab0e73834c7885d899bb89f6b60d, State: Initialized, Role: FOLLOWER
I20260808 14:39:08.760052 25376 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8115ab0e73834c7885d899bb89f6b60d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45963 } }
I20260808 14:39:08.760218 25376 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:08.760320 25376 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:08.760449 25376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:08.761574 25376 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8115ab0e73834c7885d899bb89f6b60d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45963 } }
I20260808 14:39:08.761981 25376 leader_election.cc:302] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8115ab0e73834c7885d899bb89f6b60d; no voters:
I20260808 14:39:08.762399 25376 leader_election.cc:288] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:39:08.762540 25382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:08.762775 25382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [term 1 LEADER]: Becoming Leader. State: Replica: 8115ab0e73834c7885d899bb89f6b60d, State: Running, Role: LEADER
I20260808 14:39:08.763139 25382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8115ab0e73834c7885d899bb89f6b60d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45963 } }
I20260808 14:39:08.763878 25376 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:08.764994 25383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8115ab0e73834c7885d899bb89f6b60d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8115ab0e73834c7885d899bb89f6b60d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45963 } } }
I20260808 14:39:08.765144 25383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:08.765467 25384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8115ab0e73834c7885d899bb89f6b60d. Latest consensus state: current_term: 1 leader_uuid: "8115ab0e73834c7885d899bb89f6b60d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8115ab0e73834c7885d899bb89f6b60d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45963 } } }
I20260808 14:39:08.765560 25384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:08.765738 25391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:08.769032 25391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:08.775465 25391 catalog_manager.cc:1347] Generated new cluster ID: 1144ffc28ebe402982dd3d07256093cd
I20260808 14:39:08.775559 25391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:08.794750 25391 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:39:08.796059 25391 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:08.803659 25391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8115ab0e73834c7885d899bb89f6b60d: Generated new TSK 0
I20260808 14:39:08.804563 25391 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 14:39:08.850271 25409 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 14:39:08.850474 25409 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 14:39:08.850965 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:08.823569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40020 (local address 127.19.144.126:45963)
0808 14:39:08.823735 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:08.823741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:08.823965 (+ 224us) server_negotiation.cc:407] Connection header received
0808 14:39:08.824226 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:08.824228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:08.824283 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:08.824377 (+ 94us) server_negotiation.cc:226] Negotiated authn=JWT
0808 14:39:08.824671 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:08.825216 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:08.826505 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:08.826775 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:08.826865 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:08.827338 (+ 473us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 14:39:08.850547 (+ 23209us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 14:39:08.850843 (+ 296us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40020: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 14:39:08.851083 25380 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40020: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 14:39:08.852636 25408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:08.823441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45963 (local address 127.0.0.1:40020)
0808 14:39:08.823863 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:08.823881 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:08.824032 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:08.824414 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:08.824418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:08.824427 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 14:39:08.824555 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:08.824562 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:08.825336 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:08.825339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:08.826363 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:08.826376 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:08.826902 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:08.826906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:08.827007 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:08.827073 (+ 66us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 14:39:08.850833 (+ 23760us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 14:39:08.851116 (+ 283us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.144.126:45963: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":149,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.19.144.126:45963: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 14:39:08.853466 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 25307
2026-08-08T14:39:08Z chronyd exiting
I20260808 14:39:08.893752 20033 test_util.cc:175] -----------------------------------------------
I20260808 14:39:08.893904 20033 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786199922348708-20033-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (371 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 14:39:08.895144 20033 mini_oidc.cc:152] Starting JWKS server
I20260808 14:39:08.896082 20033 webserver.cc:460] Webserver started at https://127.0.0.1:35325/ using document root <none> and password file <none>
I20260808 14:39:08.896222 20033 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 14:39:08.896549 20033 webserver.cc:460] Webserver started at http://127.0.0.1:45909/ using document root <none> and password file <none>
2026-08-08T14:39:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:08Z Disabled control of system clock
I20260808 14:39:08.908070 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:43433
--webserver_interface=127.19.144.126
--webserver_port=0
--builtin_ntp_servers=127.19.144.84:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:43433
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35325/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/testchainca.cert with env {}
W20260808 14:39:09.024706 25417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:09.025079 25417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:09.025168 25417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:09.029326 25417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:09.029452 25417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:09.029482 25417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:09.029522 25417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:09.029558 25417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 14:39:09.029597 25417 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 14:39:09.029668 25417 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35325/jwks.json
I20260808 14:39:09.035382 25417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:43433
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:43433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35325/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:09.036614 25417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:09.037784 25417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:09.047099 25422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:09.047099 25425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:09.047334 25423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:09.047127 25417 server_base.cc:1034] running on GCE node
I20260808 14:39:09.048024 25417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:09.049171 25417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:09.050377 25417 hybrid_clock.cc:648] HybridClock initialized: now 1786199949050361 us; error 31 us; skew 500 ppm
I20260808 14:39:09.052409 25417 webserver.cc:460] Webserver started at http://127.19.144.126:45349/ using document root <none> and password file <none>
I20260808 14:39:09.053071 25417 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:09.053172 25417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:09.053418 25417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:09.055656 25417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "c1f006ae0c6c4f2ea76ce74605726dcd"
format_stamp: "Formatted at 2026-08-08 14:39:09 on dist-test-slave-p3t5"
server_key: "499fce20def1d69bb013a7d20e197dfa"
server_key_iv: "38f5b163c742b98768b0bb0bce32909a"
server_key_version: "encryptionkey@0"
I20260808 14:39:09.056280 25417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "c1f006ae0c6c4f2ea76ce74605726dcd"
format_stamp: "Formatted at 2026-08-08 14:39:09 on dist-test-slave-p3t5"
server_key: "499fce20def1d69bb013a7d20e197dfa"
server_key_iv: "38f5b163c742b98768b0bb0bce32909a"
server_key_version: "encryptionkey@0"
I20260808 14:39:09.060273 25417 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 14:39:09.062970 25431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:09.064332 25417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 14:39:09.064499 25417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "c1f006ae0c6c4f2ea76ce74605726dcd"
format_stamp: "Formatted at 2026-08-08 14:39:09 on dist-test-slave-p3t5"
server_key: "499fce20def1d69bb013a7d20e197dfa"
server_key_iv: "38f5b163c742b98768b0bb0bce32909a"
server_key_version: "encryptionkey@0"
I20260808 14:39:09.064632 25417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:09.086977 25417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:09.087762 25417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:09.087992 25417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:09.095541 25417 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:43433
I20260808 14:39:09.095547 25486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:43433 every 8 connection(s)
I20260808 14:39:09.096640 25417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:09.100136 25487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:09.105145 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 25417
I20260808 14:39:09.105293 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:39:09.105646 20033 external_mini_cluster.cc:1428] Setting key 63b5e40af4dbfcb19a398df8243357d0
I20260808 14:39:09.106606 25487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd: Bootstrap starting.
I20260808 14:39:09.109153 25487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:09.110108 25487 log.cc:824] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:09.112583 25487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd: No bootstrap required, opened a new log
I20260808 14:39:09.116271 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:09.107172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43958 (local address 127.19.144.126:43433)
0808 14:39:09.107867 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:09.107887 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:09.107928 (+ 41us) server_negotiation.cc:407] Connection header received
0808 14:39:09.108576 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:09.108594 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:09.109022 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:09.109396 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:09.109835 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.111135 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.112419 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.112955 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.113078 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:09.114277 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:09.114283 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:09.114300 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:09.114862 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:09.114928 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:09.114950 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:09.115063 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:09.115184 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 14:39:09.115463 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":160,"threads_started":1}
I20260808 14:39:09.117204 25487 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f006ae0c6c4f2ea76ce74605726dcd" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43433 } }
I20260808 14:39:09.117535 25487 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:09.117620 25487 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1f006ae0c6c4f2ea76ce74605726dcd, State: Initialized, Role: FOLLOWER
I20260808 14:39:09.118122 25487 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f006ae0c6c4f2ea76ce74605726dcd" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43433 } }
I20260808 14:39:09.118315 25487 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:09.118391 25487 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:09.118494 25487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:09.119542 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:39:09.119616 25487 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f006ae0c6c4f2ea76ce74605726dcd" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43433 } }
I20260808 14:39:09.119935 25487 leader_election.cc:302] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1f006ae0c6c4f2ea76ce74605726dcd; no voters:
I20260808 14:39:09.120247 25487 leader_election.cc:288] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:39:09.120327 25492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:09.120743 25492 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [term 1 LEADER]: Becoming Leader. State: Replica: c1f006ae0c6c4f2ea76ce74605726dcd, State: Running, Role: LEADER
I20260808 14:39:09.121094 25492 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f006ae0c6c4f2ea76ce74605726dcd" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43433 } }
I20260808 14:39:09.121755 25487 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:09.122457 25495 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1f006ae0c6c4f2ea76ce74605726dcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f006ae0c6c4f2ea76ce74605726dcd" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43433 } } }
I20260808 14:39:09.122617 25495 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:09.122844 25493 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1f006ae0c6c4f2ea76ce74605726dcd. Latest consensus state: current_term: 1 leader_uuid: "c1f006ae0c6c4f2ea76ce74605726dcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f006ae0c6c4f2ea76ce74605726dcd" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43433 } } }
I20260808 14:39:09.122982 25493 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:09.123083 25499 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:09.126595 25499 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:09.132808 25499 catalog_manager.cc:1347] Generated new cluster ID: 9f4a93c3ba824e18a634fa71ea9a9e1a
I20260808 14:39:09.132899 25499 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:09.144981 25499 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:39:09.146061 25499 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:09.153014 25499 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1f006ae0c6c4f2ea76ce74605726dcd: Generated new TSK 0
I20260808 14:39:09.153805 25499 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 14:39:09.184514 25519 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 14:39:09.184638 25519 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 14:39:09.185449 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:09.168744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43966 (local address 127.19.144.126:43433)
0808 14:39:09.168907 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:09.168914 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:09.169134 (+ 220us) server_negotiation.cc:407] Connection header received
0808 14:39:09.169348 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:09.169352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:09.169439 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:09.169564 (+ 125us) server_negotiation.cc:226] Negotiated authn=JWT
0808 14:39:09.169865 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.170455 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.171367 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.171600 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.171710 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:09.172072 (+ 362us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 14:39:09.184963 (+ 12891us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 14:39:09.185279 (+ 316us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43966: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 14:39:09.185530 25490 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43966: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 14:39:09.185566 25518 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:09.168668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43433 (local address 127.0.0.1:43966)
0808 14:39:09.169041 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:09.169053 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:09.169187 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:09.169569 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:09.169573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:09.169582 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 14:39:09.169743 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:09.169749 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.170578 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.170582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:09.171231 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:09.171239 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.171732 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.171736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:09.171841 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:09.171867 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 14:39:09.185185 (+ 13318us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 14:39:09.185368 (+ 183us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.144.126:43433: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":116,"threads_started":1}
I20260808 14:39:09.186479 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 25417
2026-08-08T14:39:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (333 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 14:39:09.228518 20033 mini_oidc.cc:152] Starting JWKS server
I20260808 14:39:09.229400 20033 webserver.cc:460] Webserver started at https://127.0.0.1:43777/ using document root <none> and password file <none>
I20260808 14:39:09.229489 20033 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 14:39:09.229807 20033 webserver.cc:460] Webserver started at http://127.0.0.1:34865/ using document root <none> and password file <none>
2026-08-08T14:39:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:09Z Disabled control of system clock
I20260808 14:39:09.240576 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:39375
--webserver_interface=127.19.144.126
--webserver_port=0
--builtin_ntp_servers=127.19.144.84:45105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:39375
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43777/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 14:39:09.356348 25525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:09.356772 25525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:09.356854 25525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:09.362226 25525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:09.362325 25525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:09.362352 25525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:09.362401 25525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:09.362453 25525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 14:39:09.362493 25525 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 14:39:09.362524 25525 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43777/jwks.json
I20260808 14:39:09.367671 25525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:45105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:39375
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:39375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43777/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:09.368896 25525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:09.370452 25525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:09.379673 25531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:09.379738 25533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:09.379747 25530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:09.379827 25525 server_base.cc:1034] running on GCE node
I20260808 14:39:09.380648 25525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:09.381901 25525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:09.383085 25525 hybrid_clock.cc:648] HybridClock initialized: now 1786199949383061 us; error 33 us; skew 500 ppm
I20260808 14:39:09.385300 25525 webserver.cc:460] Webserver started at http://127.19.144.126:33719/ using document root <none> and password file <none>
I20260808 14:39:09.386040 25525 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:09.386114 25525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:09.386420 25525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:09.388828 25525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "84cb7a7d627440b6ab41e8f8e898d536"
format_stamp: "Formatted at 2026-08-08 14:39:09 on dist-test-slave-p3t5"
server_key: "f57cc8aea9bda4837c7be2cdff5ebdc3"
server_key_iv: "7841ef0ad596951f587798b4fb10b268"
server_key_version: "encryptionkey@0"
I20260808 14:39:09.389485 25525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "84cb7a7d627440b6ab41e8f8e898d536"
format_stamp: "Formatted at 2026-08-08 14:39:09 on dist-test-slave-p3t5"
server_key: "f57cc8aea9bda4837c7be2cdff5ebdc3"
server_key_iv: "7841ef0ad596951f587798b4fb10b268"
server_key_version: "encryptionkey@0"
I20260808 14:39:09.393958 25525 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:39:09.396678 25539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:09.398200 25525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:09.398387 25525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "84cb7a7d627440b6ab41e8f8e898d536"
format_stamp: "Formatted at 2026-08-08 14:39:09 on dist-test-slave-p3t5"
server_key: "f57cc8aea9bda4837c7be2cdff5ebdc3"
server_key_iv: "7841ef0ad596951f587798b4fb10b268"
server_key_version: "encryptionkey@0"
I20260808 14:39:09.398532 25525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:09.422626 25525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:09.423532 25525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:09.423794 25525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:09.432489 25594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:39375 every 8 connection(s)
I20260808 14:39:09.432490 25525 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:39375
I20260808 14:39:09.433702 25525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:09.437108 25595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:09.437554 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 25525
I20260808 14:39:09.437723 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:39:09.438058 20033 external_mini_cluster.cc:1428] Setting key df56e28483978ea95651c8e7d57497e9
I20260808 14:39:09.444413 25595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536: Bootstrap starting.
I20260808 14:39:09.447201 25595 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:09.448211 25595 log.cc:824] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:09.449239 25599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:09.439613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36098 (local address 127.19.144.126:39375)
0808 14:39:09.440383 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:09.440402 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:09.440451 (+ 49us) server_negotiation.cc:407] Connection header received
0808 14:39:09.441289 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:09.441314 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:09.441869 (+ 555us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:09.442254 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:09.442723 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.444225 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.445261 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.446038 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.446198 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:09.447090 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:09.447100 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:09.447121 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:09.447660 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:09.447734 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:09.447763 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:09.447869 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:09.448029 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 14:39:09.448323 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":134,"threads_started":1}
I20260808 14:39:09.450680 25595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536: No bootstrap required, opened a new log
I20260808 14:39:09.452316 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:39:09.454824 25595 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84cb7a7d627440b6ab41e8f8e898d536" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39375 } }
I20260808 14:39:09.455093 25595 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:09.455173 25595 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84cb7a7d627440b6ab41e8f8e898d536, State: Initialized, Role: FOLLOWER
I20260808 14:39:09.455655 25595 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84cb7a7d627440b6ab41e8f8e898d536" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39375 } }
I20260808 14:39:09.455821 25595 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:09.455910 25595 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:09.456013 25595 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:09.457013 25595 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84cb7a7d627440b6ab41e8f8e898d536" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39375 } }
I20260808 14:39:09.457386 25595 leader_election.cc:302] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84cb7a7d627440b6ab41e8f8e898d536; no voters:
I20260808 14:39:09.457765 25595 leader_election.cc:288] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:39:09.457926 25601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:09.458168 25601 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [term 1 LEADER]: Becoming Leader. State: Replica: 84cb7a7d627440b6ab41e8f8e898d536, State: Running, Role: LEADER
I20260808 14:39:09.458498 25601 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84cb7a7d627440b6ab41e8f8e898d536" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39375 } }
I20260808 14:39:09.459267 25595 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:09.460419 25603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84cb7a7d627440b6ab41e8f8e898d536. Latest consensus state: current_term: 1 leader_uuid: "84cb7a7d627440b6ab41e8f8e898d536" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84cb7a7d627440b6ab41e8f8e898d536" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39375 } } }
I20260808 14:39:09.460569 25603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:09.460872 25602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84cb7a7d627440b6ab41e8f8e898d536" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84cb7a7d627440b6ab41e8f8e898d536" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39375 } } }
I20260808 14:39:09.460968 25602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:09.461107 25610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:09.464500 25610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:09.471218 25610 catalog_manager.cc:1347] Generated new cluster ID: 1f974131eec443be8efe737404b9b2d9
I20260808 14:39:09.471333 25610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:09.488976 25610 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:39:09.490353 25610 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:09.502138 25610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84cb7a7d627440b6ab41e8f8e898d536: Generated new TSK 0
I20260808 14:39:09.503037 25610 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:39:09.525772 25599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:09.519389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36112 (local address 127.19.144.126:39375)
0808 14:39:09.519578 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:09.519584 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:09.519830 (+ 246us) server_negotiation.cc:407] Connection header received
0808 14:39:09.519948 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:09.519951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:09.520006 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:09.520099 (+ 93us) server_negotiation.cc:226] Negotiated authn=JWT
0808 14:39:09.520444 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.521115 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.522444 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.522674 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.522771 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:09.523147 (+ 376us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 14:39:09.525338 (+ 2191us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43777/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 14:39:09.525589 (+ 251us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36112: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43777/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":82}
W20260808 14:39:09.525861 25599 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36112: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43777/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 14:39:09.527647 25625 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:09.519297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:39375 (local address 127.0.0.1:36112)
0808 14:39:09.519671 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:09.519688 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:09.519819 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:09.520124 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:09.520127 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:09.520136 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 14:39:09.520281 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:09.520288 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.521249 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.521253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:09.522287 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:09.522299 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.522802 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.522806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:09.522900 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:09.522932 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 14:39:09.525528 (+ 2596us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43777/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 14:39:09.527449 (+ 1921us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.144.126:39375: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43777/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":1683,"thread_start_us":155,"threads_started":1}
I20260808 14:39:09.528326 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 25525
E20260808 14:39:09.537060 25626 webserver.cc:533] Webserver: sslize failed:
E20260808 14:39:09.537149 25626 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T14:39:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (341 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T14:39:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:09Z Disabled control of system clock
I20260808 14:39:09.580302 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:43427
--webserver_interface=127.19.144.126
--webserver_port=0
--builtin_ntp_servers=127.19.144.84:45903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:43427
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/testchainca.cert with env {}
W20260808 14:39:09.699684 25630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:09.700001 25630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:09.700043 25630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:09.704248 25630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:09.704335 25630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:09.704351 25630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:09.704368 25630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:09.704383 25630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 14:39:09.704437 25630 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 14:39:09.704471 25630 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 14:39:09.710474 25630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:45903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:43427
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:43427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:09.711692 25630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:09.712831 25630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:09.721877 25637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:09.721917 25630 server_base.cc:1034] running on GCE node
W20260808 14:39:09.721865 25638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:09.721865 25640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:09.722728 25630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:09.723905 25630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:09.725106 25630 hybrid_clock.cc:648] HybridClock initialized: now 1786199949725085 us; error 33 us; skew 500 ppm
I20260808 14:39:09.727494 25630 webserver.cc:460] Webserver started at http://127.19.144.126:36645/ using document root <none> and password file <none>
I20260808 14:39:09.728135 25630 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:09.728237 25630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:09.728490 25630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:09.730758 25630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "76fcdd5e5000481ea9f972b8189c8a2d"
format_stamp: "Formatted at 2026-08-08 14:39:09 on dist-test-slave-p3t5"
server_key: "09fbf3b062ac58a782cd84cdb81775eb"
server_key_iv: "078d8345ea6b4d4bc030fe692be50b52"
server_key_version: "encryptionkey@0"
I20260808 14:39:09.731360 25630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "76fcdd5e5000481ea9f972b8189c8a2d"
format_stamp: "Formatted at 2026-08-08 14:39:09 on dist-test-slave-p3t5"
server_key: "09fbf3b062ac58a782cd84cdb81775eb"
server_key_iv: "078d8345ea6b4d4bc030fe692be50b52"
server_key_version: "encryptionkey@0"
I20260808 14:39:09.735440 25630 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:39:09.738152 25646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:09.739604 25630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:09.739775 25630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "76fcdd5e5000481ea9f972b8189c8a2d"
format_stamp: "Formatted at 2026-08-08 14:39:09 on dist-test-slave-p3t5"
server_key: "09fbf3b062ac58a782cd84cdb81775eb"
server_key_iv: "078d8345ea6b4d4bc030fe692be50b52"
server_key_version: "encryptionkey@0"
I20260808 14:39:09.739908 25630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:09.759224 25630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:09.759999 25630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:09.760211 25630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:09.768098 25630 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:43427
I20260808 14:39:09.768136 25700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:43427 every 8 connection(s)
I20260808 14:39:09.769157 25630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:09.772403 25701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:09.777590 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 25630
I20260808 14:39:09.777819 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:39:09.778120 20033 external_mini_cluster.cc:1428] Setting key 23d1d99a4886728da8e7aee7923d5fc1
I20260808 14:39:09.778499 25701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d: Bootstrap starting.
I20260808 14:39:09.781067 25701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:09.782033 25701 log.cc:824] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:09.784458 25701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d: No bootstrap required, opened a new log
I20260808 14:39:09.788239 25701 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fcdd5e5000481ea9f972b8189c8a2d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43427 } }
I20260808 14:39:09.788496 25701 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:09.788565 25701 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76fcdd5e5000481ea9f972b8189c8a2d, State: Initialized, Role: FOLLOWER
I20260808 14:39:09.788712 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:09.779628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53520 (local address 127.19.144.126:43427)
0808 14:39:09.780325 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:09.780341 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:09.780378 (+ 37us) server_negotiation.cc:407] Connection header received
0808 14:39:09.781062 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:09.781081 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:09.781576 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:09.781957 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:09.782384 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.783360 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.784618 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.785140 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.785268 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:09.786281 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:09.786287 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:09.786302 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:09.786743 (+ 441us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:09.786804 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:09.786827 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:09.786910 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:09.787086 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 14:39:09.787424 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":189,"threads_started":1}
I20260808 14:39:09.789129 25701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fcdd5e5000481ea9f972b8189c8a2d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43427 } }
I20260808 14:39:09.789266 25701 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:09.789356 25701 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:09.789459 25701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:09.790761 25701 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fcdd5e5000481ea9f972b8189c8a2d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43427 } }
I20260808 14:39:09.791184 25701 leader_election.cc:302] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76fcdd5e5000481ea9f972b8189c8a2d; no voters:
I20260808 14:39:09.791610 25701 leader_election.cc:288] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:39:09.791728 25706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:09.791976 25706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [term 1 LEADER]: Becoming Leader. State: Replica: 76fcdd5e5000481ea9f972b8189c8a2d, State: Running, Role: LEADER
I20260808 14:39:09.792326 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:39:09.792320 25706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fcdd5e5000481ea9f972b8189c8a2d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43427 } }
I20260808 14:39:09.793038 25701 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:09.793962 25709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76fcdd5e5000481ea9f972b8189c8a2d. Latest consensus state: current_term: 1 leader_uuid: "76fcdd5e5000481ea9f972b8189c8a2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fcdd5e5000481ea9f972b8189c8a2d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43427 } } }
I20260808 14:39:09.794123 25709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:09.794497 25708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76fcdd5e5000481ea9f972b8189c8a2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76fcdd5e5000481ea9f972b8189c8a2d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43427 } } }
I20260808 14:39:09.794642 25708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:09.795431 25716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:09.798748 25716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:09.804418 25716 catalog_manager.cc:1347] Generated new cluster ID: c0effca9ed604084a09b22c106be2df8
I20260808 14:39:09.804498 25716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:09.835378 25716 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:39:09.836525 25716 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:09.845854 25716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76fcdd5e5000481ea9f972b8189c8a2d: Generated new TSK 0
I20260808 14:39:09.846815 25716 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:39:09.944003 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:09.867496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53528 (local address 127.19.144.126:43427)
0808 14:39:09.867650 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:09.867656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:09.867819 (+ 163us) server_negotiation.cc:407] Connection header received
0808 14:39:09.868020 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:09.868023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:09.868079 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:09.868171 (+ 92us) server_negotiation.cc:226] Negotiated authn=JWT
0808 14:39:09.868448 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.869026 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.870351 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.870580 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.870668 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:09.871048 (+ 380us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 14:39:09.943492 (+ 72444us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 14:39:09.943829 (+ 337us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53528: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 14:39:09.944105 25704 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53528: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 14:39:09.944196 25732 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:09.867328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43427 (local address 127.0.0.1:53528)
0808 14:39:09.867704 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:09.867721 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:09.867857 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:09.868199 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:09.868202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:09.868211 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 14:39:09.868335 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:09.868341 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.869166 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.869169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:09.870203 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:09.870215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:09.870717 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:09.870721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:09.870820 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:09.870851 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 14:39:09.943735 (+ 72884us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 14:39:09.943992 (+ 257us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.144.126:43427: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":134,"threads_started":1}
I20260808 14:39:09.944931 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 25630
2026-08-08T14:39:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (397 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (133 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (134 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:39:10 dist-test-slave-p3t5 krb5kdc[25745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:39:10 dist-test-slave-p3t5 krb5kdc[25745](info): set up 2 sockets
Aug 08 14:39:10 dist-test-slave-p3t5 krb5kdc[25745](info): commencing operation
krb5kdc: starting...
W20260808 14:39:11.286121 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:39:11 dist-test-slave-p3t5 krb5kdc[25745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:39:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:39:11.433410 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:37665
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:38495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:37665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:11.564921 25759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:11.565232 25759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:11.565276 25759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:11.569458 25759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:11.569542 25759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:11.569561 25759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:11.569581 25759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:11.569595 25759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:39:11.575194 25759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:38495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:37665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:37665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:11.576373 25759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:11.577458 25759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:11.586479 25765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:11.586501 25764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:11.586752 25759 server_base.cc:1034] running on GCE node
W20260808 14:39:11.586483 25767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:11.587417 25759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:11.588593 25759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:11.589752 25759 hybrid_clock.cc:648] HybridClock initialized: now 1786199951589718 us; error 51 us; skew 500 ppm
Aug 08 14:39:11 dist-test-slave-p3t5 krb5kdc[25745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199951, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:11.592679 25759 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:39:11.593737 25759 webserver.cc:460] Webserver started at http://127.19.144.126:45751/ using document root <none> and password file <none>
I20260808 14:39:11.594305 25759 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:11.594367 25759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:11.594538 25759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:11.596612 25759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "b83c6688aa424450aee6387bfef75fca"
format_stamp: "Formatted at 2026-08-08 14:39:11 on dist-test-slave-p3t5"
server_key: "57f54d6a94132369c346410baa05e078"
server_key_iv: "eb9e3e247e429115e9e6175e91d9df34"
server_key_version: "encryptionkey@0"
I20260808 14:39:11.597160 25759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "b83c6688aa424450aee6387bfef75fca"
format_stamp: "Formatted at 2026-08-08 14:39:11 on dist-test-slave-p3t5"
server_key: "57f54d6a94132369c346410baa05e078"
server_key_iv: "eb9e3e247e429115e9e6175e91d9df34"
server_key_version: "encryptionkey@0"
I20260808 14:39:11.601190 25759 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:39:11.603813 25775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:11.605294 25759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:39:11.605480 25759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "b83c6688aa424450aee6387bfef75fca"
format_stamp: "Formatted at 2026-08-08 14:39:11 on dist-test-slave-p3t5"
server_key: "57f54d6a94132369c346410baa05e078"
server_key_iv: "eb9e3e247e429115e9e6175e91d9df34"
server_key_version: "encryptionkey@0"
I20260808 14:39:11.605669 25759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:11.622083 25759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:11.625952 25759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:11.626192 25759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:11.633786 25759 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:37665
I20260808 14:39:11.633809 25828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:37665 every 8 connection(s)
I20260808 14:39:11.634860 25759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:11.637977 25829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:11.642859 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 25759
I20260808 14:39:11.643033 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:39:11.643332 20033 external_mini_cluster.cc:1428] Setting key 7ddf6740be390943e96c6b21802fca52
I20260808 14:39:11.644152 25829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca: Bootstrap starting.
I20260808 14:39:11.647147 25829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:11.648134 25829 log.cc:824] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:11.650468 25829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca: No bootstrap required, opened a new log
Aug 08 14:39:11 dist-test-slave-p3t5 krb5kdc[25745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:11.654397 25829 raft_consensus.cc:348] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83c6688aa424450aee6387bfef75fca" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37665 } }
I20260808 14:39:11.654676 25829 raft_consensus.cc:374] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:11.654760 25829 raft_consensus.cc:729] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b83c6688aa424450aee6387bfef75fca, State: Initialized, Role: FOLLOWER
I20260808 14:39:11.655345 25829 consensus_queue.cc:260] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83c6688aa424450aee6387bfef75fca" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37665 } }
I20260808 14:39:11.655534 25829 raft_consensus.cc:388] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:11.655632 25829 raft_consensus.cc:482] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:11.655751 25829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:11.656733 25829 raft_consensus.cc:504] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83c6688aa424450aee6387bfef75fca" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37665 } }
I20260808 14:39:11.657116 25829 leader_election.cc:302] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b83c6688aa424450aee6387bfef75fca; no voters:
I20260808 14:39:11.657534 25829 leader_election.cc:288] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:39:11.657660 25834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:11.657878 25834 raft_consensus.cc:686] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [term 1 LEADER]: Becoming Leader. State: Replica: b83c6688aa424450aee6387bfef75fca, State: Running, Role: LEADER
I20260808 14:39:11.658201 25834 consensus_queue.cc:237] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83c6688aa424450aee6387bfef75fca" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37665 } }
I20260808 14:39:11.658990 25829 sys_catalog.cc:564] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:11.660559 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:11.644904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40330 (local address 127.19.144.126:37665)
0808 14:39:11.645398 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:11.645422 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:11.645470 (+ 48us) server_negotiation.cc:407] Connection header received
0808 14:39:11.646406 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:11.646433 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:11.646937 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:11.647379 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:11.648178 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:11.649598 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:11.650670 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:11.651236 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:11.651437 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:11.654389 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:11.654395 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:11.654412 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:11.654444 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:11.656592 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:11.657163 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:11.657168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:11.657173 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:11.657344 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:11.657766 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:11.657769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:11.657771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:11.658358 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:11.658578 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:11.659008 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 14:39:11.659521 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":123,"threads_started":1}
I20260808 14:39:11.661865 25836 sys_catalog.cc:455] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [sys.catalog]: SysCatalogTable state changed. Reason: New leader b83c6688aa424450aee6387bfef75fca. Latest consensus state: current_term: 1 leader_uuid: "b83c6688aa424450aee6387bfef75fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83c6688aa424450aee6387bfef75fca" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37665 } } }
I20260808 14:39:11.662001 25836 sys_catalog.cc:458] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:11.661947 25835 sys_catalog.cc:455] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b83c6688aa424450aee6387bfef75fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83c6688aa424450aee6387bfef75fca" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37665 } } }
I20260808 14:39:11.662060 25835 sys_catalog.cc:458] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca [sys.catalog]: This master's current role is: LEADER
W20260808 14:39:11.664726 25849 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:39:11.664801 25849 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:39:11.664915 25850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:11.665721 25850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:11.671298 25850 catalog_manager.cc:1347] Generated new cluster ID: 73ad4f522935447e9d42daf281a375ee
I20260808 14:39:11.671382 25850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:11.680541 25850 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:39:11.681854 25850 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:11.703341 25850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b83c6688aa424450aee6387bfef75fca: Generated new TSK 0
I20260808 14:39:11.704313 25850 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:39:11.745877 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:37665
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:38495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:11.866767 25855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:11.867055 25855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:11.867100 25855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:11.871300 25855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:11.871366 25855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:11.871428 25855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:39:11.876929 25855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:38495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:37665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:11.878173 25855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:11.879292 25855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:11.889369 25861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:11.889361 25862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:11.889361 25865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:11.889842 25855 server_base.cc:1034] running on GCE node
I20260808 14:39:11.890298 25855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:11.891016 25855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:11.892212 25855 hybrid_clock.cc:648] HybridClock initialized: now 1786199951892192 us; error 35 us; skew 500 ppm
Aug 08 14:39:11 dist-test-slave-p3t5 krb5kdc[25745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199951, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:11.899541 25855 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:39:11.900554 25855 webserver.cc:460] Webserver started at http://127.19.144.65:34165/ using document root <none> and password file <none>
I20260808 14:39:11.901139 25855 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:11.901202 25855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:11.901381 25855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:11.903726 25855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "61dd65665f3f46609be2023835bd37f8"
format_stamp: "Formatted at 2026-08-08 14:39:11 on dist-test-slave-p3t5"
server_key: "3557331416d6e41c74e1c4c4a38d1618"
server_key_iv: "5ee3f89e12b0399650bf6e1465812ac1"
server_key_version: "encryptionkey@0"
I20260808 14:39:11.904281 25855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "61dd65665f3f46609be2023835bd37f8"
format_stamp: "Formatted at 2026-08-08 14:39:11 on dist-test-slave-p3t5"
server_key: "3557331416d6e41c74e1c4c4a38d1618"
server_key_iv: "5ee3f89e12b0399650bf6e1465812ac1"
server_key_version: "encryptionkey@0"
I20260808 14:39:11.908252 25855 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:39:11.911077 25872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:11.912417 25855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 14:39:11.912563 25855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "61dd65665f3f46609be2023835bd37f8"
format_stamp: "Formatted at 2026-08-08 14:39:11 on dist-test-slave-p3t5"
server_key: "3557331416d6e41c74e1c4c4a38d1618"
server_key_iv: "5ee3f89e12b0399650bf6e1465812ac1"
server_key_version: "encryptionkey@0"
I20260808 14:39:11.912685 25855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:11.933350 25855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:11.937371 25855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:11.937561 25855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:11.938308 25855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:11.939294 25855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:11.939349 25855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:11.939396 25855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:11.939414 25855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:11.949520 25855 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:33459
I20260808 14:39:11.949542 25989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:33459 every 8 connection(s)
I20260808 14:39:11.950608 25855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:39:11.954532 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 25855
I20260808 14:39:11.954694 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:39:11.955114 20033 external_mini_cluster.cc:1428] Setting key 1f7d193e3cfcce365ecbeeee89a73c32
Aug 08 14:39:11 dist-test-slave-p3t5 krb5kdc[25745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199951, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:39:11.967702 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:11.952730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:43779 (local address 127.19.144.126:37665)
0808 14:39:11.952891 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:11.952895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:11.953965 (+ 1070us) server_negotiation.cc:407] Connection header received
0808 14:39:11.955873 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:11.955876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:11.955926 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:11.956046 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:11.957752 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:11.958278 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:11.959904 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:11.960321 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:11.960465 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:11.963509 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:11.963516 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:11.963520 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:11.963558 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:11.965308 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:11.965960 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:11.965967 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:11.965970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:11.966075 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:11.966461 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:11.966467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:11.966470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:11.966740 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:11.966843 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:11.967455 (+ 612us) server_negotiation.cc:299] Negotiation successful
0808 14:39:11.967570 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:39:11.968622 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:11.953253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:37665 (local address 127.19.144.65:43779)
0808 14:39:11.953801 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:11.953850 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:11.955613 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:11.956233 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:11.956241 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:11.956792 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:11.957548 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:11.957575 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:11.958421 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:11.958427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:11.959780 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:11.959790 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:11.960483 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:11.960485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:11.960703 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:11.961373 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:11.961398 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:11.963300 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:11.965450 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:11.965455 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:11.965470 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:11.965828 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:11.966176 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:11.966178 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:11.966179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:11.966322 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:11.966846 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:11.966851 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:11.967194 (+ 343us) client_negotiation.cc:769] Sending connection context
0808 14:39:11.967362 (+ 168us) client_negotiation.cc:240] Negotiation successful
0808 14:39:11.967652 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":125,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
I20260808 14:39:11.970084 25991 heartbeater.cc:347] Connected to a master server at 127.19.144.126:37665
I20260808 14:39:11.970392 25991 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:11.971047 25991 heartbeater.cc:511] Master 127.19.144.126:37665 requested a full tablet report, sending...
I20260808 14:39:11.972780 25792 ts_manager.cc:194] Registered new tserver with Master: 61dd65665f3f46609be2023835bd37f8 (127.19.144.65:33459)
I20260808 14:39:11.974406 25792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:43779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:39:11.988790 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:37665
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:38495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:12.107344 25996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:12.107726 25996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:12.107831 25996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:12.112283 25996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:12.112389 25996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:12.112488 25996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:39:12.118580 25996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:38495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:37665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:12.119764 25996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:12.120889 25996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:12.131170 26002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:12.131172 26004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:12.131170 26001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:12.131495 25996 server_base.cc:1034] running on GCE node
I20260808 14:39:12.131992 25996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:12.132704 25996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:12.133890 25996 hybrid_clock.cc:648] HybridClock initialized: now 1786199952133869 us; error 32 us; skew 500 ppm
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[25745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199952, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:12.136816 25996 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:39:12.137862 25996 webserver.cc:460] Webserver started at http://127.19.144.66:37383/ using document root <none> and password file <none>
I20260808 14:39:12.138469 25996 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:12.138566 25996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:12.138814 25996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:12.140967 25996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "ba5c08a121fa4c6bb4271de3cfe021ce"
format_stamp: "Formatted at 2026-08-08 14:39:12 on dist-test-slave-p3t5"
server_key: "fbddfd7faa1eb1e0f48155a6079ffcdc"
server_key_iv: "eaf57193e7b56683540d6e0164ccd36f"
server_key_version: "encryptionkey@0"
I20260808 14:39:12.141574 25996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "ba5c08a121fa4c6bb4271de3cfe021ce"
format_stamp: "Formatted at 2026-08-08 14:39:12 on dist-test-slave-p3t5"
server_key: "fbddfd7faa1eb1e0f48155a6079ffcdc"
server_key_iv: "eaf57193e7b56683540d6e0164ccd36f"
server_key_version: "encryptionkey@0"
I20260808 14:39:12.145463 25996 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 14:39:12.148083 26012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:12.149420 25996 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:39:12.149567 25996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "ba5c08a121fa4c6bb4271de3cfe021ce"
format_stamp: "Formatted at 2026-08-08 14:39:12 on dist-test-slave-p3t5"
server_key: "fbddfd7faa1eb1e0f48155a6079ffcdc"
server_key_iv: "eaf57193e7b56683540d6e0164ccd36f"
server_key_version: "encryptionkey@0"
I20260808 14:39:12.149742 25996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:12.177057 25996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:12.180157 25996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:12.180373 25996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:12.181015 25996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:12.181955 25996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:12.182030 25996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:12.182106 25996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:12.182157 25996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:12.191591 25996 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:39153
I20260808 14:39:12.191632 26129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:39153 every 8 connection(s)
I20260808 14:39:12.192559 25996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:39:12.197147 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 25996
I20260808 14:39:12.197279 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:39:12.197574 20033 external_mini_cluster.cc:1428] Setting key d1f7d75580349bcadeab7f8c2db5d6f6
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[25745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199952, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:12.207883 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:12.194614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:51865 (local address 127.19.144.126:37665)
0808 14:39:12.194763 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:12.194767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:12.195602 (+ 835us) server_negotiation.cc:407] Connection header received
0808 14:39:12.197057 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:12.197060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:12.197125 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:12.197238 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:12.198672 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:12.199328 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:12.200508 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:12.200738 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:12.200867 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:12.203665 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:12.203673 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:12.203677 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:12.203710 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:12.205453 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:12.206174 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:12.206177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:12.206178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:12.206256 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:12.206683 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:12.206685 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:12.206686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:12.206876 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:12.206950 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:12.207649 (+ 699us) server_negotiation.cc:299] Negotiation successful
0808 14:39:12.207757 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
I20260808 14:39:12.209265 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:12.194949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:37665 (local address 127.19.144.66:51865)
0808 14:39:12.195439 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:12.195485 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:12.196807 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:12.197438 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:12.197445 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:12.197912 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:12.198520 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:12.198536 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:12.199471 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:12.199474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:12.200376 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:12.200386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:12.200913 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:12.200917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:12.201092 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:12.201764 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:12.201782 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:12.203458 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:12.205579 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:12.205589 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:12.205609 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:12.206066 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:12.206353 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:12.206357 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:12.206359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:12.206575 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:12.206992 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:12.207000 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:12.207365 (+ 365us) client_negotiation.cc:769] Sending connection context
0808 14:39:12.207605 (+ 240us) client_negotiation.cc:240] Negotiation successful
0808 14:39:12.208003 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":122,"threads_started":1}
I20260808 14:39:12.210660 26130 heartbeater.cc:347] Connected to a master server at 127.19.144.126:37665
I20260808 14:39:12.210947 26130 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:12.211490 26130 heartbeater.cc:511] Master 127.19.144.126:37665 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
I20260808 14:39:12.212744 25792 ts_manager.cc:194] Registered new tserver with Master: ba5c08a121fa4c6bb4271de3cfe021ce (127.19.144.66:39153)
I20260808 14:39:12.213358 25792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:51865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:39:12.228925 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:37665
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:38495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:12.348268 26135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:12.348553 26135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:12.348598 26135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:12.352912 26135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:12.352994 26135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:12.353087 26135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:39:12.358814 26135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:38495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:37665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:12.360092 26135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:12.361290 26135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:12.371433 26140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:12.371444 26143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:12.371408 26141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:12.371805 26135 server_base.cc:1034] running on GCE node
I20260808 14:39:12.372373 26135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:12.373250 26135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:12.374413 26135 hybrid_clock.cc:648] HybridClock initialized: now 1786199952374371 us; error 60 us; skew 500 ppm
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[25745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199952, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:12.377542 26135 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:39:12.378576 26135 webserver.cc:460] Webserver started at http://127.19.144.67:32887/ using document root <none> and password file <none>
I20260808 14:39:12.379174 26135 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:12.379238 26135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:12.379479 26135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:12.381671 26135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "edc03c298ded408e8dcb48b432835e42"
format_stamp: "Formatted at 2026-08-08 14:39:12 on dist-test-slave-p3t5"
server_key: "ac96936f7ca70d139f8329ec29d5a5a3"
server_key_iv: "546d5e188c77dce67413f75102126896"
server_key_version: "encryptionkey@0"
I20260808 14:39:12.382303 26135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "edc03c298ded408e8dcb48b432835e42"
format_stamp: "Formatted at 2026-08-08 14:39:12 on dist-test-slave-p3t5"
server_key: "ac96936f7ca70d139f8329ec29d5a5a3"
server_key_iv: "546d5e188c77dce67413f75102126896"
server_key_version: "encryptionkey@0"
I20260808 14:39:12.386366 26135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 14:39:12.389043 26151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:12.390405 26135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:12.390558 26135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "edc03c298ded408e8dcb48b432835e42"
format_stamp: "Formatted at 2026-08-08 14:39:12 on dist-test-slave-p3t5"
server_key: "ac96936f7ca70d139f8329ec29d5a5a3"
server_key_iv: "546d5e188c77dce67413f75102126896"
server_key_version: "encryptionkey@0"
I20260808 14:39:12.390688 26135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:12.411032 26135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:12.414489 26135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:12.414706 26135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:12.415395 26135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:12.416301 26135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:12.416376 26135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:12.416447 26135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:12.416497 26135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:12.426057 26135 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:40877
I20260808 14:39:12.426077 26269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:40877 every 8 connection(s)
I20260808 14:39:12.427069 26135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[25745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199952, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:12.437173 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 26135
I20260808 14:39:12.437314 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:39:12.437664 20033 external_mini_cluster.cc:1428] Setting key 86bcb945568d2739b5a903c603ff8f89
I20260808 14:39:12.441480 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:12.428974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:58467 (local address 127.19.144.126:37665)
0808 14:39:12.429153 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:12.429158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:12.430134 (+ 976us) server_negotiation.cc:407] Connection header received
0808 14:39:12.431534 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:12.431537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:12.431587 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:12.431665 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:12.432959 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:12.433500 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:12.434726 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:12.434949 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:12.435032 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:12.437921 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:12.437925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:12.437928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:12.437957 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:12.439636 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:12.440153 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:12.440156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:12.440157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:12.440239 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:12.440530 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:12.440531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:12.440533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:12.440699 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:12.440768 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:12.441255 (+ 487us) server_negotiation.cc:299] Negotiation successful
0808 14:39:12.441357 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 14:39:12.442389 26273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:12.429396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:37665 (local address 127.19.144.67:58467)
0808 14:39:12.429961 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:12.430009 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:12.431318 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:12.431878 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:12.431885 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:12.432278 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:12.432804 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:12.432820 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:12.433613 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:12.433636 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:12.434601 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:12.434612 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:12.435090 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:12.435093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:12.435248 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:12.435972 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:12.435989 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:12.437730 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:12.439770 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:12.439777 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:12.439794 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:12.440063 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:12.440320 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:12.440322 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:12.440323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:12.440469 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:12.440781 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:12.440790 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:12.441036 (+ 246us) client_negotiation.cc:769] Sending connection context
0808 14:39:12.441189 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 14:39:12.441465 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":153,"threads_started":1}
I20260808 14:39:12.443612 26271 heartbeater.cc:347] Connected to a master server at 127.19.144.126:37665
I20260808 14:39:12.443907 26271 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:12.444429 26271 heartbeater.cc:511] Master 127.19.144.126:37665 requested a full tablet report, sending...
I20260808 14:39:12.445473 25792 ts_manager.cc:194] Registered new tserver with Master: edc03c298ded408e8dcb48b432835e42 (127.19.144.67:40877)
I20260808 14:39:12.446099 25792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:58467
I20260808 14:39:12.451809 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/test-admin.keytab.
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[25745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:12.471131 20033 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 14:39:12.474269 26278 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:12.472664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50370 (local address 127.19.144.65:33459)
0808 14:39:12.473005 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:12.473015 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:12.473047 (+ 32us) server_negotiation.cc:407] Connection header received
0808 14:39:12.473108 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:12.473114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:12.473322 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:12.473447 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:12.474061 (+ 614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50370: BlockingRecv error: recv got EOF from 127.0.0.1:50370 (error 108)
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":158,"threads_started":1}
W20260808 14:39:12.474393 20033 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/krb5cc' of type FILE
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[25745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:12.475517 20033 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[25745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:12.476486 20033 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[25745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.65@KRBTEST.COM
I20260808 14:39:12.486608 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:12.477401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50374 (local address 127.19.144.65:33459)
0808 14:39:12.477548 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:12.477551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:12.477569 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:39:12.477753 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:12.477756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:12.477801 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:12.477870 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:12.478417 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:12.479173 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:12.479989 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:12.480366 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:12.480454 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:12.482925 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:12.482938 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:12.482954 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:12.482979 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:12.484648 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:12.485159 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:12.485165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:12.485166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:12.485252 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:12.485496 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:12.485498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:12.485499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:12.485825 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:12.486004 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:12.486236 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 14:39:12.486399 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 14:39:12.490672 26278 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:12.489754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50382 (local address 127.19.144.65:33459)
0808 14:39:12.489892 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:12.489895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:12.489922 (+ 27us) server_negotiation.cc:407] Connection header received
0808 14:39:12.490034 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:12.490036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:12.490070 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:12.490134 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:12.490561 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50382: BlockingRecv error: recv got EOF from 127.0.0.1:50382 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 14:39:12.490753 20033 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/krb5cc' of type FILE
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[25745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:12.491797 20033 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[25745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:12.492693 20033 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[25745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.65@KRBTEST.COM
I20260808 14:39:12.501688 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:12.493587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50394 (local address 127.19.144.65:33459)
0808 14:39:12.493741 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:12.493745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:12.493865 (+ 120us) server_negotiation.cc:407] Connection header received
0808 14:39:12.493961 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:12.493963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:12.493998 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:12.494064 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:12.494688 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:12.495198 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:12.495964 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:12.496175 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:12.496254 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:12.498565 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:12.498569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:12.498571 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:12.498595 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:12.499897 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:12.500370 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:12.500374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:12.500376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:12.500467 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:12.500751 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:12.500755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:12.500756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:12.500947 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:12.501110 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:12.501335 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 14:39:12.501466 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 14:39:12.504657 26278 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:12.503543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50406 (local address 127.19.144.65:33459)
0808 14:39:12.503683 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:12.503687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:12.503803 (+ 116us) server_negotiation.cc:407] Connection header received
0808 14:39:12.503984 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:12.503988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:12.504034 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:12.504116 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:12.504530 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50406: BlockingRecv error: recv got EOF from 127.0.0.1:50406 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 14:39:12.504886 20033 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786199922348708-20033-0/krb5kdc/krb5cc' of type FILE
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[25745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:12.506011 20033 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[25745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:12.506906 20033 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[25745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.65@KRBTEST.COM
I20260808 14:39:12.515715 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:12.507725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50412 (local address 127.19.144.65:33459)
0808 14:39:12.507857 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:12.507861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:12.507981 (+ 120us) server_negotiation.cc:407] Connection header received
0808 14:39:12.508127 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:12.508130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:12.508176 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:12.508261 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:12.508765 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:12.509266 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:12.510158 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:12.510378 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:12.510498 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:12.512758 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:12.512764 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:12.512771 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:12.512795 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:12.514057 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:12.514480 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:12.514484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:12.514488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:12.514568 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:12.514856 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:12.514859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:12.514861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:12.515037 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:12.515190 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:12.515428 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 14:39:12.515551 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:39:12.516805 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 25855
I20260808 14:39:12.524732 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 25996
I20260808 14:39:12.531323 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 26135
I20260808 14:39:12.538795 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 25759
2026-08-08T14:39:12Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2327 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[26291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[26291](info): set up 2 sockets
Aug 08 14:39:12 dist-test-slave-p3t5 krb5kdc[26291](info): commencing operation
krb5kdc: starting...
W20260808 14:39:13.607784 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:39:13 dist-test-slave-p3t5 krb5kdc[26291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:39:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:13Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.126.keytab.
Entry for principal oryx/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.126.keytab.
I20260808 14:39:13.752525 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:46629
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.126.keytab
--principal=oryx/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:46629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:13.871338 26305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:13.871657 26305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:13.871698 26305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:13.875770 26305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:13.875836 26305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:13.875855 26305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:13.875872 26305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:13.875888 26305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:39:13.881268 26305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:46629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.126.keytab
--principal=oryx/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:46629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:13.882385 26305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:13.883389 26305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:39:13.892544 26305 server_base.cc:1034] running on GCE node
W20260808 14:39:13.892477 26313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:13.892477 26311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:13.892514 26310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:13.893579 26305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:13.894865 26305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:13.896057 26305 hybrid_clock.cc:648] HybridClock initialized: now 1786199953896041 us; error 33 us; skew 500 ppm
Aug 08 14:39:13 dist-test-slave-p3t5 krb5kdc[26291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199953, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:13.899214 26305 init.cc:377] Logged in from keytab as oryx/127.19.144.126@KRBTEST.COM (short username oryx)
I20260808 14:39:13.900280 26305 webserver.cc:460] Webserver started at http://127.19.144.126:38161/ using document root <none> and password file <none>
I20260808 14:39:13.900906 26305 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:13.900975 26305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:13.901213 26305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:13.903494 26305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "0828373b803c4ef5a62d163e24b66108"
format_stamp: "Formatted at 2026-08-08 14:39:13 on dist-test-slave-p3t5"
server_key: "c0590c0d0f066ae348325d9059365046"
server_key_iv: "9535ad711dd79882dd634f05e974278d"
server_key_version: "encryptionkey@0"
I20260808 14:39:13.904111 26305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "0828373b803c4ef5a62d163e24b66108"
format_stamp: "Formatted at 2026-08-08 14:39:13 on dist-test-slave-p3t5"
server_key: "c0590c0d0f066ae348325d9059365046"
server_key_iv: "9535ad711dd79882dd634f05e974278d"
server_key_version: "encryptionkey@0"
I20260808 14:39:13.908102 26305 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 14:39:13.910761 26320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:13.912065 26305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:13.912209 26305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "0828373b803c4ef5a62d163e24b66108"
format_stamp: "Formatted at 2026-08-08 14:39:13 on dist-test-slave-p3t5"
server_key: "c0590c0d0f066ae348325d9059365046"
server_key_iv: "9535ad711dd79882dd634f05e974278d"
server_key_version: "encryptionkey@0"
I20260808 14:39:13.912329 26305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:13.925693 26305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:13.928855 26305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:13.929083 26305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:13.936723 26305 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:46629
I20260808 14:39:13.937083 26376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:46629 every 8 connection(s)
I20260808 14:39:13.937885 26305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:13.940529 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 26305
I20260808 14:39:13.940675 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:39:13.941042 20033 external_mini_cluster.cc:1428] Setting key ea732627252c40c9621877ba731c7a6c
I20260808 14:39:13.941208 26377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:13.949100 26377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108: Bootstrap starting.
Aug 08 14:39:13 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM
I20260808 14:39:13.952173 26377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:13.953143 26377 log.cc:824] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:13.955281 26377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108: No bootstrap required, opened a new log
I20260808 14:39:13.957134 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:13.942623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36994 (local address 127.19.144.126:46629)
0808 14:39:13.943051 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:13.943077 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:13.943117 (+ 40us) server_negotiation.cc:407] Connection header received
0808 14:39:13.944112 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:13.944132 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:13.944494 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:13.944792 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:13.945841 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:13.947262 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:13.948503 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:13.949023 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:13.949160 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:13.952159 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:13.952165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:13.952181 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:13.952213 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:13.954138 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:13.954636 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:13.954641 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:13.954646 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:13.954767 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:13.955075 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:13.955078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:13.955080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:13.955561 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:13.955771 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:13.956080 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 14:39:13.956387 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":144,"threads_started":1}
I20260808 14:39:13.959342 26377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } }
I20260808 14:39:13.959671 26377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:13.959750 26377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0828373b803c4ef5a62d163e24b66108, State: Initialized, Role: FOLLOWER
I20260808 14:39:13.960153 26377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } }
I20260808 14:39:13.960273 26377 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:13.960317 26377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:13.960390 26377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:13.961285 26377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } }
I20260808 14:39:13.961594 26377 leader_election.cc:302] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0828373b803c4ef5a62d163e24b66108; no voters:
I20260808 14:39:13.961971 26377 leader_election.cc:288] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:39:13.962132 26382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:13.962393 26382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [term 1 LEADER]: Becoming Leader. State: Replica: 0828373b803c4ef5a62d163e24b66108, State: Running, Role: LEADER
I20260808 14:39:13.962719 26382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } }
I20260808 14:39:13.963289 26377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:13.964718 26383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } }
I20260808 14:39:13.964736 26384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0828373b803c4ef5a62d163e24b66108. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } }
I20260808 14:39:13.964870 26383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:13.964874 26384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:13.965353 26392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:13.968495 26392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:13.974342 26392 catalog_manager.cc:1347] Generated new cluster ID: df6391f5877f42caa44d19deb4f08545
I20260808 14:39:13.974761 26392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:13.994997 26392 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:39:13.996052 26392 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:14.003026 26392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108: Generated new TSK 0
I20260808 14:39:14.003816 26392 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.65.keytab.
Entry for principal oryx/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.65.keytab.
I20260808 14:39:14.041591 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46629
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.65.keytab
--principal=oryx/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:14.161420 26404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:14.161794 26404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:14.161849 26404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:14.166178 26404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:14.166245 26404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:14.166349 26404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:39:14.171783 26404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.65.keytab
--principal=oryx/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:14.172943 26404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:14.174042 26404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:14.184095 26411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:14.184183 26404 server_base.cc:1034] running on GCE node
W20260808 14:39:14.184080 26412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:14.184079 26414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:14.184948 26404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:14.185667 26404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:14.186883 26404 hybrid_clock.cc:648] HybridClock initialized: now 1786199954186871 us; error 31 us; skew 500 ppm
Aug 08 14:39:14 dist-test-slave-p3t5 krb5kdc[26291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199954, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:14.189874 26404 init.cc:377] Logged in from keytab as oryx/127.19.144.65@KRBTEST.COM (short username oryx)
I20260808 14:39:14.190832 26404 webserver.cc:460] Webserver started at http://127.19.144.65:46763/ using document root <none> and password file <none>
I20260808 14:39:14.191391 26404 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:14.191454 26404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:14.191627 26404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:14.193774 26404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "90167cc5e982452a89df283e189e1d92"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "b6eb1bc02494639217301a668d0b6f84"
server_key_iv: "d22bc251c854410906fb792f8c13d91b"
server_key_version: "encryptionkey@0"
I20260808 14:39:14.194329 26404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "90167cc5e982452a89df283e189e1d92"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "b6eb1bc02494639217301a668d0b6f84"
server_key_iv: "d22bc251c854410906fb792f8c13d91b"
server_key_version: "encryptionkey@0"
I20260808 14:39:14.198333 26404 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:39:14.200825 26421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:14.202208 26404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:14.202343 26404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "90167cc5e982452a89df283e189e1d92"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "b6eb1bc02494639217301a668d0b6f84"
server_key_iv: "d22bc251c854410906fb792f8c13d91b"
server_key_version: "encryptionkey@0"
I20260808 14:39:14.202461 26404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:14.224823 26404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:14.228289 26404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:14.228540 26404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:14.229254 26404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:14.230247 26404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:14.230297 26404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:14.230372 26404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:14.230403 26404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:14.239708 26404 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:43199
I20260808 14:39:14.239728 26542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:43199 every 8 connection(s)
I20260808 14:39:14.240720 26404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:39:14.250039 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 26404
I20260808 14:39:14.250205 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:39:14.250556 20033 external_mini_cluster.cc:1428] Setting key 9cc131ea0ebe49b83d1a304ca72145ae
Aug 08 14:39:14 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199954, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM
I20260808 14:39:14.255729 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:14.242691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:34217 (local address 127.19.144.126:46629)
0808 14:39:14.242845 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:14.242849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:14.243814 (+ 965us) server_negotiation.cc:407] Connection header received
0808 14:39:14.245107 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:14.245110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:14.245162 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:14.245246 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:14.246564 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:14.247125 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:14.248277 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:14.248515 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:14.248597 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:14.251577 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:14.251581 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:14.251583 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:14.251608 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:14.253260 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:14.254000 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:14.254003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:14.254004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:14.254133 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:14.254520 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:14.254526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:14.254530 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:14.254800 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:14.254912 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:14.255441 (+ 529us) server_negotiation.cc:299] Negotiation successful
0808 14:39:14.255597 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 14:39:14.256523 26545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:14.243158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.65:34217)
0808 14:39:14.243636 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:14.243683 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:14.244868 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:14.245422 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:14.245438 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:14.245903 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:14.246405 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:14.246420 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:14.247253 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:14.247256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:14.248160 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:14.248169 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:14.248649 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:14.248652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:14.248802 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:14.249467 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:14.249484 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:14.251390 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:14.253391 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:14.253402 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:14.253423 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:14.253899 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:14.254256 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:14.254259 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:14.254260 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:14.254410 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:14.254908 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:14.254914 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:14.255168 (+ 254us) client_negotiation.cc:769] Sending connection context
0808 14:39:14.255327 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 14:39:14.255601 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":123,"threads_started":1}
I20260808 14:39:14.257982 26543 heartbeater.cc:347] Connected to a master server at 127.19.144.126:46629
I20260808 14:39:14.258270 26543 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:14.258929 26543 heartbeater.cc:511] Master 127.19.144.126:46629 requested a full tablet report, sending...
I20260808 14:39:14.260818 26338 ts_manager.cc:194] Registered new tserver with Master: 90167cc5e982452a89df283e189e1d92 (127.19.144.65:43199)
WARNING: no policy specified for oryx/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:39:14.262565 26338 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:34217
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.144.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.66.keytab.
Entry for principal oryx/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.66.keytab.
I20260808 14:39:14.283004 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46629
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.66.keytab
--principal=oryx/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:14.404031 26549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:14.404357 26549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:14.404404 26549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:14.408857 26549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:14.408939 26549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:14.409045 26549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:39:14.414601 26549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.66.keytab
--principal=oryx/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:14.415917 26549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:14.417129 26549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:14.428063 26554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:14.428068 26557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:14.428071 26555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:14.428257 26549 server_base.cc:1034] running on GCE node
I20260808 14:39:14.429088 26549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:14.429972 26549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:14.431166 26549 hybrid_clock.cc:648] HybridClock initialized: now 1786199954431155 us; error 48 us; skew 500 ppm
Aug 08 14:39:14 dist-test-slave-p3t5 krb5kdc[26291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199954, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:14.434497 26549 init.cc:377] Logged in from keytab as oryx/127.19.144.66@KRBTEST.COM (short username oryx)
I20260808 14:39:14.435566 26549 webserver.cc:460] Webserver started at http://127.19.144.66:46021/ using document root <none> and password file <none>
I20260808 14:39:14.436234 26549 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:14.436337 26549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:14.436591 26549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:14.438922 26549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "29ecf3b442034a2bb6a52f6177787f43"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "d7005e0bbbfc0996c28c6b12bd555250"
server_key_iv: "c2775eccd40e0c1b9a0c45ee85196d48"
server_key_version: "encryptionkey@0"
I20260808 14:39:14.439545 26549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "29ecf3b442034a2bb6a52f6177787f43"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "d7005e0bbbfc0996c28c6b12bd555250"
server_key_iv: "c2775eccd40e0c1b9a0c45ee85196d48"
server_key_version: "encryptionkey@0"
I20260808 14:39:14.443648 26549 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:39:14.446381 26564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:14.447831 26549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:14.447989 26549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "29ecf3b442034a2bb6a52f6177787f43"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "d7005e0bbbfc0996c28c6b12bd555250"
server_key_iv: "c2775eccd40e0c1b9a0c45ee85196d48"
server_key_version: "encryptionkey@0"
I20260808 14:39:14.448130 26549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:14.475852 26549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:14.479207 26549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:14.479444 26549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:14.480115 26549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:14.481133 26549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:14.481211 26549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:14.481288 26549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:14.481343 26549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:14.491072 26549 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:41105
I20260808 14:39:14.491093 26683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:41105 every 8 connection(s)
I20260808 14:39:14.492165 26549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:39:14.501255 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 26549
I20260808 14:39:14.501400 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:39:14.501770 20033 external_mini_cluster.cc:1428] Setting key fd2a742191d623bce8a64138977f787a
Aug 08 14:39:14 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199954, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM
I20260808 14:39:14.508152 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:14.494100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:53987 (local address 127.19.144.126:46629)
0808 14:39:14.494272 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:14.494277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:14.495176 (+ 899us) server_negotiation.cc:407] Connection header received
0808 14:39:14.496578 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:14.496581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:14.496633 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:14.496715 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:14.498443 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:14.498978 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:14.500229 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:14.500465 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:14.500548 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:14.503868 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:14.503872 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:14.503874 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:14.503900 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:14.505553 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:14.506298 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:14.506302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:14.506303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:14.506389 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:14.506867 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:14.506869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:14.506870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:14.507141 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:14.507221 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:14.507910 (+ 689us) server_negotiation.cc:299] Negotiation successful
0808 14:39:14.508028 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 14:39:14.509537 26686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:14.494542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.66:53987)
0808 14:39:14.495014 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:14.495063 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:14.496359 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:14.497045 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:14.497054 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:14.497533 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:14.498268 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:14.498286 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:14.499123 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:14.499126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:14.500101 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:14.500112 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:14.500611 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:14.500614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:14.500762 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:14.501596 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:14.501671 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:14.503641 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:14.505713 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:14.505723 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:14.505744 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:14.506185 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:14.506488 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:14.506493 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:14.506498 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:14.506707 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:14.507255 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:14.507264 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:14.507638 (+ 374us) client_negotiation.cc:769] Sending connection context
0808 14:39:14.507881 (+ 243us) client_negotiation.cc:240] Negotiation successful
0808 14:39:14.508284 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":110,"threads_started":1}
I20260808 14:39:14.511013 26684 heartbeater.cc:347] Connected to a master server at 127.19.144.126:46629
I20260808 14:39:14.511303 26684 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:14.511852 26684 heartbeater.cc:511] Master 127.19.144.126:46629 requested a full tablet report, sending...
I20260808 14:39:14.513077 26338 ts_manager.cc:194] Registered new tserver with Master: 29ecf3b442034a2bb6a52f6177787f43 (127.19.144.66:41105)
WARNING: no policy specified for oryx/127.19.144.67@KRBTEST.COM; defaulting to no policy
I20260808 14:39:14.513819 26338 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:53987
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.144.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.67.keytab.
Entry for principal oryx/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.67.keytab.
I20260808 14:39:14.535341 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46629
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.67.keytab
--principal=oryx/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:14.660485 26690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:14.660841 26690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:14.660941 26690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:14.665385 26690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:14.665498 26690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:14.665596 26690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:39:14.671303 26690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.67.keytab
--principal=oryx/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:14.672555 26690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:14.673796 26690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:14.684316 26695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:14.684363 26696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:14.684705 26690 server_base.cc:1034] running on GCE node
W20260808 14:39:14.684326 26698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:14.685297 26690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:14.686113 26690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:14.687326 26690 hybrid_clock.cc:648] HybridClock initialized: now 1786199954687290 us; error 58 us; skew 500 ppm
Aug 08 14:39:14 dist-test-slave-p3t5 krb5kdc[26291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199954, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:14.690542 26690 init.cc:377] Logged in from keytab as oryx/127.19.144.67@KRBTEST.COM (short username oryx)
I20260808 14:39:14.691609 26690 webserver.cc:460] Webserver started at http://127.19.144.67:42571/ using document root <none> and password file <none>
I20260808 14:39:14.692272 26690 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:14.692370 26690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:14.692618 26690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:14.695067 26690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "7424b0259e6997b97afe73d5ef7eed8a"
server_key_iv: "583d0144c27e1cafabce1e9a4aa4d52b"
server_key_version: "encryptionkey@0"
I20260808 14:39:14.695742 26690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "7424b0259e6997b97afe73d5ef7eed8a"
server_key_iv: "583d0144c27e1cafabce1e9a4aa4d52b"
server_key_version: "encryptionkey@0"
I20260808 14:39:14.700176 26690 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:39:14.703156 26705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:14.704553 26690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 14:39:14.704712 26690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "7424b0259e6997b97afe73d5ef7eed8a"
server_key_iv: "583d0144c27e1cafabce1e9a4aa4d52b"
server_key_version: "encryptionkey@0"
I20260808 14:39:14.704846 26690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:14.728006 26690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:14.731974 26690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:14.732213 26690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:14.732926 26690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:14.733955 26690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:14.734030 26690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:14.734103 26690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:14.734157 26690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:14.743644 26824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:38887 every 8 connection(s)
I20260808 14:39:14.743644 26690 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:38887
I20260808 14:39:14.744715 26690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:39:14.754065 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 26690
I20260808 14:39:14.754232 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:39:14.754583 20033 external_mini_cluster.cc:1428] Setting key 5e0e9a0fb443bd9350d459ffc554c7a0
Aug 08 14:39:14 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199954, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM
I20260808 14:39:14.759531 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:14.746737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:45155 (local address 127.19.144.126:46629)
0808 14:39:14.746916 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:14.746922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:14.747828 (+ 906us) server_negotiation.cc:407] Connection header received
0808 14:39:14.749182 (+ 1354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:14.749185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:14.749236 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:14.749332 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:14.750672 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:14.751228 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:14.752493 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:14.752735 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:14.752897 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:14.755765 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:14.755770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:14.755772 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:14.755804 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:14.757537 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:14.758097 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:14.758100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:14.758101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:14.758183 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:14.758502 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:14.758505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:14.758506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:14.758693 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:14.758768 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:14.759288 (+ 520us) server_negotiation.cc:299] Negotiation successful
0808 14:39:14.759403 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 14:39:14.760480 26827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:14.747199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.67:45155)
0808 14:39:14.747665 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:14.747711 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:14.748967 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:14.749507 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:14.749514 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:14.749996 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:14.750507 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:14.750521 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:14.751361 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:14.751363 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:14.752371 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:14.752381 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:14.752912 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:14.752916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:14.753086 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:14.753816 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:14.753833 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:14.755613 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:14.757686 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:14.757696 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:14.757711 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:14.758005 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:14.758265 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:14.758267 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:14.758268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:14.758410 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:14.758780 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:14.758786 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:14.759056 (+ 270us) client_negotiation.cc:769] Sending connection context
0808 14:39:14.759218 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 14:39:14.759533 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":109,"threads_started":1}
I20260808 14:39:14.761945 26825 heartbeater.cc:347] Connected to a master server at 127.19.144.126:46629
I20260808 14:39:14.762217 26825 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:14.762768 26825 heartbeater.cc:511] Master 127.19.144.126:46629 requested a full tablet report, sending...
I20260808 14:39:14.763957 26338 ts_manager.cc:194] Registered new tserver with Master: bd3bf41b5d504ecba9db5ca6ac52cef7 (127.19.144.67:38887)
I20260808 14:39:14.764662 26338 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:45155
I20260808 14:39:14.769012 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.19.144.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.144.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.144.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.125.keytab.
Entry for principal oryx/127.19.144.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.125.keytab.
I20260808 14:39:14.800851 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.125:35749
--webserver_interface=127.19.144.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.125.keytab
--principal=oryx/127.19.144.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:46629,127.19.144.125:35749
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:14.922192 26830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:14.922518 26830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:14.922560 26830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:14.926676 26830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:14.926772 26830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:14.926800 26830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:14.926821 26830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:14.926842 26830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:39:14.932865 26830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:46629,127.19.144.125:35749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.125.keytab
--principal=oryx/127.19.144.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.125:35749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.144.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
W20260808 14:39:14.933106 26830 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.19.144.126:46629,127.19.144.125:35749'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 14:39:14.934098 26830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:14.935191 26830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:14.944429 26838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:14.944517 26836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:14.944432 26835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:14.945086 26830 server_base.cc:1034] running on GCE node
I20260808 14:39:14.945979 26830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:14.947366 26830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:14.948559 26830 hybrid_clock.cc:648] HybridClock initialized: now 1786199954948517 us; error 57 us; skew 500 ppm
Aug 08 14:39:14 dist-test-slave-p3t5 krb5kdc[26291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199954, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:14.951581 26830 init.cc:377] Logged in from keytab as oryx/127.19.144.125@KRBTEST.COM (short username oryx)
I20260808 14:39:14.952629 26830 webserver.cc:460] Webserver started at http://127.19.144.125:37657/ using document root <none> and password file <none>
I20260808 14:39:14.953194 26830 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:14.953255 26830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:14.953433 26830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:14.955636 26830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/data/instance:
uuid: "4cab1eb43c6042ceadf9392eb38e6a2b"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "76f9d83e729d00ff5ef4f1c2e11d1ef7"
server_key_iv: "8b618b799e26fb010fee02942f662ed3"
server_key_version: "encryptionkey@0"
I20260808 14:39:14.956190 26830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/wal/instance:
uuid: "4cab1eb43c6042ceadf9392eb38e6a2b"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "76f9d83e729d00ff5ef4f1c2e11d1ef7"
server_key_iv: "8b618b799e26fb010fee02942f662ed3"
server_key_version: "encryptionkey@0"
I20260808 14:39:14.960137 26830 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:39:14.962841 26845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:14.964359 26830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:14.964496 26830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/wal
uuid: "4cab1eb43c6042ceadf9392eb38e6a2b"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "76f9d83e729d00ff5ef4f1c2e11d1ef7"
server_key_iv: "8b618b799e26fb010fee02942f662ed3"
server_key_version: "encryptionkey@0"
I20260808 14:39:14.964620 26830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:14.978050 26830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:14.981179 26830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:14.981403 26830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:14.989094 26830 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.125:35749
I20260808 14:39:14.989415 26901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.125:35749 every 8 connection(s)
I20260808 14:39:14.990617 26830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/data/info.pb
I20260808 14:39:14.993729 26902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:14.997687 26902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:39:14.998719 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 26830
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199954, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.125@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.144.125@KRBTEST.COM
I20260808 14:39:15.014318 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:14.999844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37004 (local address 127.19.144.126:46629)
0808 14:39:14.999994 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:14.999999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.001124 (+ 1125us) server_negotiation.cc:407] Connection header received
0808 14:39:15.001963 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.001967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.002025 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.002141 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.003332 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.004263 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.005948 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.006311 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.006432 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.009934 (+ 3502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.009937 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.009939 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.009966 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.011729 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.012355 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.012358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.012359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.012441 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.012924 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.012929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.012931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.013266 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.013373 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.013981 (+ 608us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.014122 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:39:15.015156 26905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.000318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58182 (local address 127.19.144.125:35749)
0808 14:39:15.000811 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.000828 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.000866 (+ 38us) server_negotiation.cc:407] Connection header received
0808 14:39:15.001728 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.001750 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.002019 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.002215 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.003154 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.004578 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.005941 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.006580 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.006668 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.010390 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.010397 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.010411 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.010435 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.012116 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.012605 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.012610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.012611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.012736 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.013128 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.013131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.013133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.013587 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.013811 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.014054 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.014262 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":332,"spinlock_wait_cycles":12416,"thread_start_us":171,"threads_started":1}
I20260808 14:39:15.015734 26906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.000318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:37004)
0808 14:39:15.000942 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.000996 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.001732 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.002234 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.002239 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.002731 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.003153 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.003172 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.004438 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.004441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.005800 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.005822 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.006481 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.006484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.006669 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.007574 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.007602 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.009738 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.011863 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.011871 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.011883 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.012231 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.012541 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.012546 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.012552 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.012712 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.013380 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.013386 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.013585 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.013877 (+ 292us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.015549 (+ 1672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":1329,"spinlock_wait_cycles":3840,"thread_start_us":175,"threads_started":1}
I20260808 14:39:15.017462 26902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199954, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.125@KRBTEST.COM for oryx/127.19.144.125@KRBTEST.COM
I20260808 14:39:15.026418 26905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.018137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58198 (local address 127.19.144.125:35749)
0808 14:39:15.018262 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.018265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.018279 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:39:15.018353 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.018355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.018390 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.018454 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.018803 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.019312 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.020062 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.020860 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.020953 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.023114 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.023117 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.023119 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.023139 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.024656 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.025116 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.025120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.025121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.025198 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.025477 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.025480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.025482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.025734 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.025850 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.026093 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.026254 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":52}
I20260808 14:39:15.026418 26906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.018065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:35749 (local address 127.0.0.1:58198)
0808 14:39:15.018176 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.018191 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.018272 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.018462 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.018464 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.018607 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.018703 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.018708 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.019411 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.019413 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.019955 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.019964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.020975 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.020978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.021063 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.021561 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.021572 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.022967 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.024807 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.024810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.024812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.025010 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.025292 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.025295 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.025297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.025381 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.025837 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.025840 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.025937 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.026046 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.026196 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 14:39:15.029774 26902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: Bootstrap starting.
I20260808 14:39:15.032173 26902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:15.032922 26902 log.cc:824] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:15.035120 26902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: No bootstrap required, opened a new log
I20260808 14:39:15.038362 26902 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } }
I20260808 14:39:15.038604 26902 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:15.038641 26902 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cab1eb43c6042ceadf9392eb38e6a2b, State: Initialized, Role: FOLLOWER
I20260808 14:39:15.039149 26902 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } }
I20260808 14:39:15.039683 26908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } } }
I20260808 14:39:15.039835 26908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:39:15.040483 26902 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: configured and running, proceeding with master startup.
W20260808 14:39:15.045518 26919 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:39:15.045653 26919 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:39:15.047952 26830 master_runner.cc:428] Detected that this master 4cab1eb43c6042ceadf9392eb38e6a2b is joining an existing cluster
I20260808 14:39:15.048041 26830 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.144.125:35749
I20260808 14:39:15.059419 26905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.050855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58200 (local address 127.19.144.125:35749)
0808 14:39:15.050969 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.050972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.051141 (+ 169us) server_negotiation.cc:407] Connection header received
0808 14:39:15.051282 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.051285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.051326 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.051405 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.051983 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.052497 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.053428 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.053696 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.053800 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.055200 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.055203 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.055205 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.055226 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.056952 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.057473 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.057477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.057479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.057570 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.058563 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.058568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.058570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.058754 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.058858 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.059115 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.059270 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":44}
I20260808 14:39:15.059442 26926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.050734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:35749 (local address 127.0.0.1:58200)
0808 14:39:15.051057 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.051068 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.051178 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.051435 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.051438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.051712 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.051872 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.051878 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.052647 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.052651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.053302 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.053311 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.053800 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.053803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.053902 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.054467 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.054479 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.054846 (+ 367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.057072 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.057076 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.057078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.057346 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.058352 (+ 1006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.058355 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.058357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.058466 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.058857 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.058860 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.058954 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.059058 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.059215 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":4,"thread_start_us":101,"threads_started":1}
I20260808 14:39:15.060305 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.050359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37008 (local address 127.19.144.126:46629)
0808 14:39:15.050501 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.050505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.050781 (+ 276us) server_negotiation.cc:407] Connection header received
0808 14:39:15.051049 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.051053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.051110 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.051195 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.051971 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.052797 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.054057 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.054420 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.054498 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.056355 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.056360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.056363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.056397 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.058191 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.058686 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.058690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.058692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.058785 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.059102 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.059105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.059107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.059331 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.059401 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.060007 (+ 606us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.060130 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 14:39:15.060856 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.050284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:37008)
0808 14:39:15.050665 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.050703 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.050876 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.051336 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.051340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.051638 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.051827 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.051836 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.052939 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.052942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.053917 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.053928 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.054657 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.054660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.054757 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.055703 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.055718 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.056195 (+ 477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.058307 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.058310 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.058312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.058565 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.058902 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.058908 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.058911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.058998 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.059475 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.059478 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.059817 (+ 339us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.060510 (+ 693us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.060668 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":21,"thread_start_us":109,"threads_started":1}
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199953, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.126@KRBTEST.COM for oryx/127.19.144.125@KRBTEST.COM
I20260808 14:39:15.076522 26905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.065993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58204 (local address 127.19.144.125:35749)
0808 14:39:15.066149 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.066154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.066484 (+ 330us) server_negotiation.cc:407] Connection header received
0808 14:39:15.066714 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.066730 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.066785 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.066879 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.067596 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.068386 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.069665 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.069896 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.070025 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.072967 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.072970 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.072972 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.072997 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.074538 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.075116 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.075118 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.075119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.075197 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.075530 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.075535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.075537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.075760 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.075839 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.076286 (+ 447us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.076389 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 14:39:15.076803 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.065964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:35749 (local address 127.0.0.1:58204)
0808 14:39:15.066348 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.066395 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.066569 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.066920 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.066927 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.067341 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.067468 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.067474 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.068500 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.068505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.069494 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.069506 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.070030 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.070034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.070155 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.070793 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.070806 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.072775 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.074661 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.074672 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.074681 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.075012 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.075301 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.075305 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.075307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.075412 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.075880 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.075888 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.076056 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.076238 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.076517 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":101,"threads_started":1}
I20260808 14:39:15.078696 26338 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.144.125:35749: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b"
member_type: NON_VOTER
last_known_addr {
host: "127.19.144.125"
port: 35749
}
attrs {
promote: true
}
}
dest_uuid: "0828373b803c4ef5a62d163e24b66108"
cas_config_opid_index: -1
I20260808 14:39:15.079468 26338 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.19.144.124@KRBTEST.COM; defaulting to no policy
I20260808 14:39:15.081669 26931 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 4cab1eb43c6042ceadf9392eb38e6a2b (127.19.144.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } } }
I20260808 14:39:15.082384 26875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:15.083730 26931 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.144.125:35749
I20260808 14:39:15.083879 26875 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 FOLLOWER]: Refusing update from remote peer 0828373b803c4ef5a62d163e24b66108: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.144.124@KRBTEST.COM" created.
I20260808 14:39:15.084461 26830 master.cc:537] Master@127.19.144.125:35749 shutting down...
I20260808 14:39:15.084218 26930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } } }
I20260808 14:39:15.084793 26930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:15.085036 26930 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 14:39:15.087777 26323 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 14:39:15.089112 26323 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 -> Peer 4cab1eb43c6042ceadf9392eb38e6a2b (127.19.144.125:35749): Couldn't send request to peer 4cab1eb43c6042ceadf9392eb38e6a2b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 14:39:15.090375 26830 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:15.090698 26830 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:15.090838 26830 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.144.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.124.keytab.
Entry for principal oryx/127.19.144.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.124.keytab.
I20260808 14:39:15.105605 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.124:36719
--webserver_interface=127.19.144.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.124.keytab
--principal=oryx/127.19.144.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:46629,127.19.144.125:35749,127.19.144.124:36719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:39:15.106910 26830 master.cc:552] Master@127.19.144.125:35749 shutdown complete.
I20260808 14:39:15.106981 26830 master_runner.cc:305] Clearing existing system tablet
I20260808 14:39:15.107273 26830 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:15.107343 26830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:15.108786 26830 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 14:39:15.109949 26937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:15.110296 26830 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:15.110452 26830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 14:39:15.110493 26830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/wal
uuid: "4cab1eb43c6042ceadf9392eb38e6a2b"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "76f9d83e729d00ff5ef4f1c2e11d1ef7"
server_key_iv: "8b618b799e26fb010fee02942f662ed3"
server_key_version: "encryptionkey@0"
I20260808 14:39:15.112522 26830 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:15.114637 26830 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 14:39:15.114758 26830 log.cc:1196] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 14:39:15.115051 26830 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: Deleting consensus metadata
I20260808 14:39:15.115262 26830 master_runner.cc:315] Copying system tablet from 127.19.144.126:46629
I20260808 14:39:15.116756 26830 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: tablet copy: Beginning tablet copy session from remote peer at address 127.19.144.126:46629
I20260808 14:39:15.125590 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.117755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37014 (local address 127.19.144.126:46629)
0808 14:39:15.117902 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.117907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.117923 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:39:15.118078 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.118082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.118141 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.118237 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.118972 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.119524 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.120386 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.120629 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.120742 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.122160 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.122164 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.122167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.122202 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.123824 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.124309 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.124313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.124315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.124404 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.124694 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.124698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.124700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.124898 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.125010 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.125295 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.125417 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 14:39:15.125599 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.117471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:37014)
0808 14:39:15.117760 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.117807 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.117938 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.118259 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.118262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.118499 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.118834 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.118841 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.119649 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.119652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.120252 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.120260 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.120757 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.120761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.120881 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.121496 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.121509 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.121982 (+ 473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.123963 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.123967 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.123969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.124187 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.124501 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.124505 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.124507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.124597 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.124998 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.125002 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.125113 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.125226 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.125384 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":113,"threads_started":1}
I20260808 14:39:15.126648 26359 tablet_copy_service.cc:140] P 0828373b803c4ef5a62d163e24b66108: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 4cab1eb43c6042ceadf9392eb38e6a2b ({username='oryx', principal='oryx/127.19.144.125@KRBTEST.COM'} at 127.0.0.1:37014)
I20260808 14:39:15.126879 26359 tablet_copy_service.cc:161] P 0828373b803c4ef5a62d163e24b66108: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 4cab1eb43c6042ceadf9392eb38e6a2b at {username='oryx', principal='oryx/127.19.144.125@KRBTEST.COM'} at 127.0.0.1:37014: session id = 4cab1eb43c6042ceadf9392eb38e6a2b-00000000000000000000000000000000
I20260808 14:39:15.129004 26359 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 14:39:15.130932 26830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 14:39:15.133163 26830 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: tablet copy: Starting download of 0 data blocks...
I20260808 14:39:15.133406 26830 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: tablet copy: Starting download of 1 WAL segments...
I20260808 14:39:15.135378 26830 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 14:39:15.137087 26359 tablet_copy_service.cc:342] P 0828373b803c4ef5a62d163e24b66108: Request end of tablet copy session 4cab1eb43c6042ceadf9392eb38e6a2b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.144.125@KRBTEST.COM'} at 127.0.0.1:37014
I20260808 14:39:15.137228 26359 tablet_copy_service.cc:433] P 0828373b803c4ef5a62d163e24b66108: ending tablet copy session 4cab1eb43c6042ceadf9392eb38e6a2b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 4cab1eb43c6042ceadf9392eb38e6a2b
I20260808 14:39:15.144595 26830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:15.146454 26945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:15.146713 26947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:15.147042 26830 server_base.cc:1034] running on GCE node
W20260808 14:39:15.149766 26944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:15.150012 26830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:15.150367 26830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:15.151508 26830 hybrid_clock.cc:648] HybridClock initialized: now 1786199955151506 us; error 57 us; skew 500 ppm
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199955, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:15.153244 26830 init.cc:377] Logged in from keytab as oryx/127.19.144.125@KRBTEST.COM (short username oryx)
I20260808 14:39:15.153831 26830 webserver.cc:460] Webserver started at http://127.19.144.125:42285/ using document root <none> and password file <none>
I20260808 14:39:15.154062 26830 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:15.154120 26830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:15.155256 26830 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 14:39:15.156245 26957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:15.156491 26830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 14:39:15.156592 26830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/wal
uuid: "4cab1eb43c6042ceadf9392eb38e6a2b"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "76f9d83e729d00ff5ef4f1c2e11d1ef7"
server_key_iv: "8b618b799e26fb010fee02942f662ed3"
server_key_version: "encryptionkey@0"
I20260808 14:39:15.156703 26830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:15.168880 26830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:15.171145 26830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:15.176185 26830 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.125:35749
I20260808 14:39:15.176821 26830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-1/data/info.pb
I20260808 14:39:15.177222 27014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.125:35749 every 8 connection(s)
I20260808 14:39:15.178623 27015 sys_catalog.cc:263] Verifying existing consensus state
I20260808 14:39:15.179481 27015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: Bootstrap starting.
I20260808 14:39:15.188545 27015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:39:15.188946 27015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: Bootstrap complete.
I20260808 14:39:15.189527 27015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } }
I20260808 14:39:15.189739 27015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4cab1eb43c6042ceadf9392eb38e6a2b, State: Initialized, Role: LEARNER
I20260808 14:39:15.189881 27015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } }
I20260808 14:39:15.190307 27019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } } }
I20260808 14:39:15.190487 27015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:15.190487 27019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: This master's current role is: LEARNER
I20260808 14:39:15.193784 27031 catalog_manager.cc:1259] Loaded cluster ID: df6391f5877f42caa44d19deb4f08545
I20260808 14:39:15.193924 27031 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: loading cluster ID for follower catalog manager: success
I20260808 14:39:15.195629 27031 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: acquiring CA information for follower catalog manager: success
I20260808 14:39:15.196791 27031 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 14:39:15.265532 26543 heartbeater.cc:503] Master 127.19.144.126:46629 was elected leader, sending a full tablet report...
W20260808 14:39:15.271528 26935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:15.271826 26935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:15.271868 26935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:15.276081 26935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:15.276191 26935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:15.276227 26935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:15.276275 26935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:15.276312 26935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:39:15.282266 26935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:46629,127.19.144.125:35749,127.19.144.124:36719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.124.keytab
--principal=oryx/127.19.144.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.124:36719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.144.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:15.283527 26935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:15.284673 26935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:15.294363 27037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:15.294349 27039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:15.294549 26935 server_base.cc:1034] running on GCE node
W20260808 14:39:15.294349 27036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:15.295411 26935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:15.296644 26935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:15.297845 26935 hybrid_clock.cc:648] HybridClock initialized: now 1786199955297828 us; error 32 us; skew 500 ppm
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199955, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:15.300933 26935 init.cc:377] Logged in from keytab as oryx/127.19.144.124@KRBTEST.COM (short username oryx)
I20260808 14:39:15.302105 26935 webserver.cc:460] Webserver started at http://127.19.144.124:34939/ using document root <none> and password file <none>
I20260808 14:39:15.302738 26935 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:15.302853 26935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:15.303110 26935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:15.305529 26935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/data/instance:
uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b"
format_stamp: "Formatted at 2026-08-08 14:39:15 on dist-test-slave-p3t5"
server_key: "533272844306c5c8b261b13ebb695c23"
server_key_iv: "5072e860641dc8a68a16e112f7e36e90"
server_key_version: "encryptionkey@0"
I20260808 14:39:15.306318 26935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/wal/instance:
uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b"
format_stamp: "Formatted at 2026-08-08 14:39:15 on dist-test-slave-p3t5"
server_key: "533272844306c5c8b261b13ebb695c23"
server_key_iv: "5072e860641dc8a68a16e112f7e36e90"
server_key_version: "encryptionkey@0"
I20260808 14:39:15.310774 26935 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:39:15.313757 27047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:15.315271 26935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:15.315480 26935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/wal
uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b"
format_stamp: "Formatted at 2026-08-08 14:39:15 on dist-test-slave-p3t5"
server_key: "533272844306c5c8b261b13ebb695c23"
server_key_iv: "5072e860641dc8a68a16e112f7e36e90"
server_key_version: "encryptionkey@0"
I20260808 14:39:15.315623 26935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:15.331862 26935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:15.335366 26935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:15.335594 26935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:15.343407 26935 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.124:36719
I20260808 14:39:15.343721 27105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.124:36719 every 8 connection(s)
I20260808 14:39:15.344563 26935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/data/info.pb
I20260808 14:39:15.345459 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 26935
I20260808 14:39:15.347983 27106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:15.352737 27106 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.144.124@KRBTEST.COM
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199955, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.124@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM
I20260808 14:39:15.361912 27108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.347048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41094 (local address 127.19.144.124:36719)
0808 14:39:15.347498 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.347517 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.347562 (+ 45us) server_negotiation.cc:407] Connection header received
0808 14:39:15.348450 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.348475 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.348893 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.349271 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.350125 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.351656 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.352722 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.353415 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.353572 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.356586 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.356592 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.356604 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.356635 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.358680 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.359195 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.359199 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.359204 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.359317 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.359636 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.359641 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.359643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.360059 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.360298 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.360539 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.360888 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"spinlock_wait_cycles":7424,"thread_start_us":129,"threads_started":1}
I20260808 14:39:15.365240 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.354961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41342 (local address 127.19.144.126:46629)
0808 14:39:15.355101 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.355107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.355392 (+ 285us) server_negotiation.cc:407] Connection header received
0808 14:39:15.355568 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.355572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.355624 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.355710 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.356472 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.357238 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.358463 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.358830 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.358917 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.361267 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.361271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.361273 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.361301 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.362964 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.363612 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.363617 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.363619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.363720 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.364084 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.364088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.364090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.364357 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.364489 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.364958 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.365100 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:39:15.365259 27109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.354827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:41342)
0808 14:39:15.355267 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.355306 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.355447 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.355769 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.355777 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.356207 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.356336 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.356342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.357381 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.357384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.358329 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.358340 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.358956 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.358960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.359094 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.359702 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.359714 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.361123 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.363075 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.363083 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.363089 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.363456 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.363844 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.363847 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.363850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.363943 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.364556 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.364561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.364736 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.364865 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.365079 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"spinlock_wait_cycles":3968,"thread_start_us":142,"threads_started":1}
I20260808 14:39:15.366362 27106 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199955, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.124@KRBTEST.COM for oryx/127.19.144.125@KRBTEST.COM
I20260808 14:39:15.375149 27109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.366874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:35749 (local address 127.0.0.1:48100)
0808 14:39:15.367015 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.367028 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.367099 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.367578 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.367581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.367758 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.367862 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.367867 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.368863 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.368866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.369404 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.369411 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.369997 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.370001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.370124 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.370601 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.370614 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.371952 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.373768 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.373771 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.373773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.373982 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.374255 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.374258 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.374259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.374344 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.374701 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.374704 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.374804 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.374894 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.375016 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 14:39:15.375206 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.367047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48100 (local address 127.19.144.125:35749)
0808 14:39:15.367331 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.367344 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.367357 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:39:15.367415 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.367417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.367457 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.367563 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.367989 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.368698 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.369545 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.369874 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.369997 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.372108 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.372112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.372114 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.372141 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.373605 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.374074 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.374076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.374078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.374158 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.374422 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.374425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.374426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.374611 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.374687 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.374961 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.375078 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":91,"threads_started":1}
I20260808 14:39:15.376166 27106 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199955, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.124@KRBTEST.COM for oryx/127.19.144.124@KRBTEST.COM
I20260808 14:39:15.384570 27108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.376754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41106 (local address 127.19.144.124:36719)
0808 14:39:15.376869 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.376876 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.376926 (+ 50us) server_negotiation.cc:407] Connection header received
0808 14:39:15.377059 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.377062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.377100 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.377191 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.377546 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.378141 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.378884 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.379574 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.379694 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.381563 (+ 1869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.381567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.381570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.381592 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.382959 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.383366 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.383371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.383373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.383462 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.383738 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.383741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.383743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.383924 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.384027 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.384296 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.384446 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":60}
I20260808 14:39:15.384570 27109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.376720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:36719 (local address 127.0.0.1:41106)
0808 14:39:15.376847 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.376861 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.376972 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.377201 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.377203 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.377369 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.377436 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.377442 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.378249 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.378253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.378754 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.378761 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.379692 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.379695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.379771 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.380217 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.380227 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.381432 (+ 1205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.383066 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.383072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.383074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.383268 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.383558 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.383561 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.383563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.383643 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.384027 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.384030 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.384118 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.384223 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.384380 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 14:39:15.389043 27106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: Bootstrap starting.
I20260808 14:39:15.391883 27106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:15.392645 27106 log.cc:824] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:15.394826 27106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: No bootstrap required, opened a new log
I20260808 14:39:15.398011 27106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } }
I20260808 14:39:15.398274 27106 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:15.398311 27106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67e5b6d0c46b4db088c2ef22d8bdd22b, State: Initialized, Role: FOLLOWER
I20260808 14:39:15.398795 27106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } }
I20260808 14:39:15.399294 27112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } } }
I20260808 14:39:15.399435 27112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:39:15.400219 27106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: configured and running, proceeding with master startup.
W20260808 14:39:15.405838 27124 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:39:15.405957 27124 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:39:15.409569 26935 master_runner.cc:428] Detected that this master 67e5b6d0c46b4db088c2ef22d8bdd22b is joining an existing cluster
I20260808 14:39:15.409680 26935 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.144.124:36719
I20260808 14:39:15.416837 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 26404
I20260808 14:39:15.432106 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:43199
--local_ip_for_outbound_sockets=127.19.144.65
--tserver_master_addrs=127.19.144.126:46629
--webserver_port=46763
--webserver_interface=127.19.144.65
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.65.keytab
--principal=oryx/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.144.126:46629,127.19.144.125:35749
--tserver_master_addrs=127.19.144.126:46629,127.19.144.125:35749,127.19.144.124:36719 with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:39:15.444152 27108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.413335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41122 (local address 127.19.144.124:36719)
0808 14:39:15.413442 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.413446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.414107 (+ 661us) server_negotiation.cc:407] Connection header received
0808 14:39:15.414237 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.414240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.414272 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.414314 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.414865 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.415601 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.418514 (+ 2913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.418990 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.419094 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.420761 (+ 1667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.420765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.420768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.420793 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.423225 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.442496 (+ 19271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.442504 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.442507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.442633 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.442991 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.442995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.442997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.443260 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.443377 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.443772 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.443920 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 14:39:15.444907 27132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.413526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:36719 (local address 127.0.0.1:41122)
0808 14:39:15.414039 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.414052 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.414154 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.414433 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.414436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.414624 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.414717 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.414723 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.415765 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.415771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.418366 (+ 2595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.418382 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.419277 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.419285 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.419426 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.420015 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.420034 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.420609 (+ 575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.423338 (+ 2729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.423341 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.423343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.442283 (+ 18940us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.442766 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.442771 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.442775 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.442873 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.443434 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.443440 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.443567 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.444443 (+ 876us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.444634 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":3,"thread_start_us":129,"threads_started":1}
I20260808 14:39:15.447263 27130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.412336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:41352)
0808 14:39:15.412647 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.412664 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.412788 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.413260 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.413266 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.413540 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.413790 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.413797 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.414909 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.414912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.423203 (+ 8291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.423218 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.424018 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.424022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.424124 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.424590 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.424604 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.442396 (+ 17792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.445447 (+ 3051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.445451 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.445453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.445747 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.446090 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.446093 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.446095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.446182 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.446680 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.446683 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.446813 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.446925 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.447078 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":7263,"thread_start_us":134,"threads_started":1}
I20260808 14:39:15.448225 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.412913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48114 (local address 127.19.144.125:35749)
0808 14:39:15.413026 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.413031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.413333 (+ 302us) server_negotiation.cc:407] Connection header received
0808 14:39:15.413528 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.413531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.413584 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.413674 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.414346 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.415173 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.416724 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.417058 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.417129 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.419461 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.419465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.419468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.419498 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.421761 (+ 2263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.443844 (+ 22083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.443850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.443852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.443975 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.447033 (+ 3058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.447037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.447039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.447294 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.447389 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.447904 (+ 515us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.448061 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 14:39:15.448486 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.412399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41352 (local address 127.19.144.126:46629)
0808 14:39:15.412534 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.412544 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.412753 (+ 209us) server_negotiation.cc:407] Connection header received
0808 14:39:15.412938 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.412942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.413006 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.413070 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.413931 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.414786 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.423371 (+ 8585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.423893 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.424021 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.442655 (+ 18634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.442660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.442664 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.442700 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.444783 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.445874 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.445878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.445880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.445982 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.446280 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.446285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.446288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.446532 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.446633 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.448181 (+ 1548us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.448316 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 14:39:15.448767 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.412930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:35749 (local address 127.0.0.1:48114)
0808 14:39:15.413242 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.413256 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.413399 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.413836 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.413839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.414076 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.414195 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.414203 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.415498 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.415501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.416520 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.416532 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.417278 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.417281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.417365 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.418368 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.418382 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.419284 (+ 902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.422215 (+ 2931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.422219 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.422224 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.443182 (+ 20958us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.446659 (+ 3477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.446662 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.446664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.446903 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.447460 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.447464 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.447567 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.448438 (+ 871us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.448596 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":28,"thread_start_us":107,"threads_started":1}
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199953, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.126@KRBTEST.COM for oryx/127.19.144.124@KRBTEST.COM
I20260808 14:39:15.463183 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.453216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:36719 (local address 127.0.0.1:41136)
0808 14:39:15.453385 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.453399 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.453483 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.453824 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.453828 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.454040 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.454152 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.454159 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.455187 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.455191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.456102 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.456112 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.456798 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.456802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.456907 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.457507 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.457525 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.459259 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.461346 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.461350 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.461352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.461644 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.461983 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.461987 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.461989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.462096 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.462560 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.462564 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.462676 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.462823 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.462992 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260808 14:39:15.463222 27108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.453327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41136 (local address 127.19.144.124:36719)
0808 14:39:15.453457 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.453461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.453476 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:39:15.453594 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.453597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.453705 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.453783 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.454274 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.455066 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.456264 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.456655 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.456766 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.459442 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.459446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.459449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.459478 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.461210 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.461770 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.461773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.461775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.461873 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.462207 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.462213 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.462217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.462453 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.462554 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.462940 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.463068 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:39:15.464774 26338 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.144.124:36719: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b"
member_type: NON_VOTER
last_known_addr {
host: "127.19.144.124"
port: 36719
}
attrs {
promote: true
}
}
dest_uuid: "0828373b803c4ef5a62d163e24b66108"
cas_config_opid_index: 5
I20260808 14:39:15.465427 26338 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } }
I20260808 14:39:15.466648 26931 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 67e5b6d0c46b4db088c2ef22d8bdd22b (127.19.144.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } } }
I20260808 14:39:15.467563 27077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:15.468101 26931 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.144.124:36719
I20260808 14:39:15.468263 26931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } } }
I20260808 14:39:15.468353 26931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:15.469102 27077 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 FOLLOWER]: Refusing update from remote peer 0828373b803c4ef5a62d163e24b66108: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 14:39:15.469960 26930 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:15.474123 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.468580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48118 (local address 127.19.144.125:35749)
0808 14:39:15.468999 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.469005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.469150 (+ 145us) server_negotiation.cc:407] Connection header received
0808 14:39:15.469346 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.469350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.469423 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.469502 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:15.469941 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.470762 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.472370 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.473072 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.473170 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.473322 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.473884 (+ 562us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.473983 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327}
I20260808 14:39:15.474577 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.466788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:35749 (local address 127.0.0.1:48118)
0808 14:39:15.469090 (+ 2302us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.469104 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.469222 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.469700 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.469703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.469721 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:15.469809 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.469815 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.470915 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.470918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.472216 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.472227 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.473425 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.473428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.473539 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.473595 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.474343 (+ 748us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.474452 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2232}
I20260808 14:39:15.475051 27077 raft_consensus.cc:858] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } }
I20260808 14:39:15.475107 26982 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 LEARNER]: Refusing update from remote peer 0828373b803c4ef5a62d163e24b66108: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20260808 14:39:15.475286 27077 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } }
I20260808 14:39:15.475545 26930 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 14:39:15.477360 27077 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 4cab1eb43c6042ceadf9392eb38e6a2b (127.19.144.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } } }
I20260808 14:39:15.477360 26982 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 67e5b6d0c46b4db088c2ef22d8bdd22b (127.19.144.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } } }
I20260808 14:39:15.479781 27112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0828373b803c4ef5a62d163e24b66108. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } } }
I20260808 14:39:15.479768 27019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0828373b803c4ef5a62d163e24b66108. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } } }
I20260808 14:39:15.479890 27112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 14:39:15.479890 27019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: This master's current role is: LEARNER
I20260808 14:39:15.479991 27112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } } }
I20260808 14:39:15.479990 27019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } } }
I20260808 14:39:15.480069 27019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: This master's current role is: LEARNER
I20260808 14:39:15.480069 27112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 14:39:15.480149 27112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } } }
I20260808 14:39:15.480223 27112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 14:39:15.480460 26935 master.cc:537] Master@127.19.144.124:36719 shutting down...
I20260808 14:39:15.484489 27134 mvcc.cc:205] Tried to move back new op lower bound from 7316275011591479296 to 7316275011432411136. Current Snapshot: MvccSnapshot[applied={T|T < 7316275011468029952}]
I20260808 14:39:15.484983 26930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: NON_VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: true } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } } }
I20260808 14:39:15.485132 26930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: This master's current role is: LEADER
W20260808 14:39:15.486481 26323 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 -> Peer 67e5b6d0c46b4db088c2ef22d8bdd22b (127.19.144.124:36719): Couldn't send request to peer 67e5b6d0c46b4db088c2ef22d8bdd22b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 14:39:15.487196 26935 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260808 14:39:15.487932 26931 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108: attempting to promote NON_VOTER 4cab1eb43c6042ceadf9392eb38e6a2b to VOTER
I20260808 14:39:15.488389 26931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } }
I20260808 14:39:15.490382 26982 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 LEARNER]: Refusing update from remote peer 0828373b803c4ef5a62d163e24b66108: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260808 14:39:15.490835 26323 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 -> Peer 67e5b6d0c46b4db088c2ef22d8bdd22b (127.19.144.124:36719): Couldn't send request to peer 67e5b6d0c46b4db088c2ef22d8bdd22b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 14:39:15.490957 27143 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 14:39:15.493204 26982 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 4cab1eb43c6042ceadf9392eb38e6a2b (127.19.144.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } } }
I20260808 14:39:15.495235 27143 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 4cab1eb43c6042ceadf9392eb38e6a2b (127.19.144.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } } }
I20260808 14:39:15.495656 26935 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260808 14:39:15.495782 26935 tablet_replica.cc:330] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: stopping tablet replica
I20260808 14:39:15.496251 27138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } } }
I20260808 14:39:15.496376 27138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:39:15.497400 27143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } } }
I20260808 14:39:15.497502 26931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } } }
I20260808 14:39:15.497663 26931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:15.497570 27143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:15.502020 26935 master.cc:552] Master@127.19.144.124:36719 shutdown complete.
I20260808 14:39:15.502077 26935 master_runner.cc:305] Clearing existing system tablet
I20260808 14:39:15.502381 26935 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:15.502444 26935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:15.503743 26935 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 14:39:15.504648 27147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:15.504923 26935 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:15.505050 26935 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 14:39:15.505084 26935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/wal
uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b"
format_stamp: "Formatted at 2026-08-08 14:39:15 on dist-test-slave-p3t5"
server_key: "533272844306c5c8b261b13ebb695c23"
server_key_iv: "5072e860641dc8a68a16e112f7e36e90"
server_key_version: "encryptionkey@0"
I20260808 14:39:15.506946 26935 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:15.509332 26935 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 14:39:15.509426 26935 log.cc:1196] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 14:39:15.509837 26935 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: Deleting consensus metadata
I20260808 14:39:15.510028 26935 master_runner.cc:315] Copying system tablet from 127.19.144.126:46629
I20260808 14:39:15.511291 26935 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: tablet copy: Beginning tablet copy session from remote peer at address 127.19.144.126:46629
I20260808 14:39:15.516440 26684 heartbeater.cc:503] Master 127.19.144.126:46629 was elected leader, sending a full tablet report...
I20260808 14:39:15.521956 27153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.511851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:41354)
0808 14:39:15.512147 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.512165 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.512285 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.512606 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.512609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.512843 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.512964 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.512973 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.514140 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.514144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.515009 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.515020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.515653 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.515657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.515771 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.517276 (+ 1505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.517292 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.517827 (+ 535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.520129 (+ 2302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.520133 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.520136 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.520437 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.520781 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.520784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.520786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.520877 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.521342 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.521345 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.521457 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.521552 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.521751 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":94,"threads_started":1}
I20260808 14:39:15.522710 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.512224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41354 (local address 127.19.144.126:46629)
0808 14:39:15.512344 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.512349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.512364 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:39:15.512412 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.512416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.512476 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.512565 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.513117 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.514013 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.515142 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.515527 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.515633 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.518137 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.518141 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.518144 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.518173 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.519991 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.520546 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.520549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.520551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.520667 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.520977 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.520980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.520982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.521211 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.521299 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.522405 (+ 1106us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.522553 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 14:39:15.523587 26359 tablet_copy_service.cc:140] P 0828373b803c4ef5a62d163e24b66108: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 67e5b6d0c46b4db088c2ef22d8bdd22b ({username='oryx', principal='oryx/127.19.144.124@KRBTEST.COM'} at 127.0.0.1:41354)
I20260808 14:39:15.523708 26359 tablet_copy_service.cc:161] P 0828373b803c4ef5a62d163e24b66108: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 67e5b6d0c46b4db088c2ef22d8bdd22b at {username='oryx', principal='oryx/127.19.144.124@KRBTEST.COM'} at 127.0.0.1:41354: session id = 67e5b6d0c46b4db088c2ef22d8bdd22b-00000000000000000000000000000000
I20260808 14:39:15.524923 26359 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 14:39:15.526409 26935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 14:39:15.528985 26935 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: tablet copy: Starting download of 0 data blocks...
I20260808 14:39:15.529196 26935 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: tablet copy: Starting download of 1 WAL segments...
I20260808 14:39:15.530992 26935 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 14:39:15.533121 26359 tablet_copy_service.cc:342] P 0828373b803c4ef5a62d163e24b66108: Request end of tablet copy session 67e5b6d0c46b4db088c2ef22d8bdd22b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.144.124@KRBTEST.COM'} at 127.0.0.1:41354
I20260808 14:39:15.533221 26359 tablet_copy_service.cc:433] P 0828373b803c4ef5a62d163e24b66108: ending tablet copy session 67e5b6d0c46b4db088c2ef22d8bdd22b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 67e5b6d0c46b4db088c2ef22d8bdd22b
I20260808 14:39:15.541786 26935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:15.543846 27154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:15.544049 27157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:15.544158 27155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:15.544296 26935 server_base.cc:1034] running on GCE node
I20260808 14:39:15.544665 26935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:15.545013 26935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:15.546180 26935 hybrid_clock.cc:648] HybridClock initialized: now 1786199955546148 us; error 49 us; skew 500 ppm
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199955, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:15.547914 26935 init.cc:377] Logged in from keytab as oryx/127.19.144.124@KRBTEST.COM (short username oryx)
I20260808 14:39:15.548458 26935 webserver.cc:460] Webserver started at http://127.19.144.124:33955/ using document root <none> and password file <none>
I20260808 14:39:15.548661 26935 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:15.548719 26935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:15.550266 26935 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 14:39:15.551491 27164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:15.551793 26935 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 14:39:15.551869 26935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/wal
uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b"
format_stamp: "Formatted at 2026-08-08 14:39:15 on dist-test-slave-p3t5"
server_key: "533272844306c5c8b261b13ebb695c23"
server_key_iv: "5072e860641dc8a68a16e112f7e36e90"
server_key_version: "encryptionkey@0"
I20260808 14:39:15.551975 26935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:15.580456 26935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:15.582778 26935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:15.588274 26935 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.124:36719
I20260808 14:39:15.588320 27221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.124:36719 every 8 connection(s)
I20260808 14:39:15.588932 26935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/master-2/data/info.pb
I20260808 14:39:15.590827 27222 sys_catalog.cc:263] Verifying existing consensus state
W20260808 14:39:15.590915 27133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:15.591357 27133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:15.591434 27133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 14:39:15.592078 27222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: Bootstrap starting.
W20260808 14:39:15.597496 27133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:15.597590 27133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:15.597707 27133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:39:15.600642 27222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:39:15.600967 27222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: Bootstrap complete.
I20260808 14:39:15.601586 27222 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } }
I20260808 14:39:15.601751 27222 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 67e5b6d0c46b4db088c2ef22d8bdd22b, State: Initialized, Role: LEARNER
I20260808 14:39:15.601868 27222 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } }
I20260808 14:39:15.602263 27225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } } }
I20260808 14:39:15.602365 27222 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:15.602471 27225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: This master's current role is: LEARNER
I20260808 14:39:15.606158 27236 catalog_manager.cc:1259] Loaded cluster ID: df6391f5877f42caa44d19deb4f08545
I20260808 14:39:15.606119 27133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.65.keytab
--principal=oryx/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:43199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=46763
--tserver_master_addrs=127.19.144.126:46629,127.19.144.125:35749,127.19.144.124:36719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:15.606246 27236 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: loading cluster ID for follower catalog manager: success
I20260808 14:39:15.607340 27133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:15.607952 27236 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: acquiring CA information for follower catalog manager: success
I20260808 14:39:15.608541 27133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:39:15.609318 27236 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 14:39:15.619661 27242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:15.619661 27239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:15.619843 27240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:15.620014 27133 server_base.cc:1034] running on GCE node
I20260808 14:39:15.620497 27133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:15.621199 27133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:15.622381 27133 hybrid_clock.cc:648] HybridClock initialized: now 1786199955622360 us; error 30 us; skew 500 ppm
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199955, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:15.625586 27133 init.cc:377] Logged in from keytab as oryx/127.19.144.65@KRBTEST.COM (short username oryx)
I20260808 14:39:15.626698 27133 webserver.cc:460] Webserver started at http://127.19.144.65:46763/ using document root <none> and password file <none>
I20260808 14:39:15.627344 27133 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:15.627460 27133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:15.631740 27133 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:39:15.634016 27249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:15.635377 27133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:15.635635 27133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "90167cc5e982452a89df283e189e1d92"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "b6eb1bc02494639217301a668d0b6f84"
server_key_iv: "d22bc251c854410906fb792f8c13d91b"
server_key_version: "encryptionkey@0"
I20260808 14:39:15.636222 27133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:15.661171 27133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:15.665140 27133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:15.665401 27133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:15.666229 27133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:15.667215 27133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:15.667268 27133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:15.667341 27133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:15.667372 27133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:15.677048 27133 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:43199
I20260808 14:39:15.677067 27366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:43199 every 8 connection(s)
I20260808 14:39:15.678241 27133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:39:15.683774 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 27133
I20260808 14:39:15.684015 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 26549
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199955, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.124@KRBTEST.COM
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199955, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.125@KRBTEST.COM
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199955, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM
I20260808 14:39:15.694423 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:41105
--local_ip_for_outbound_sockets=127.19.144.66
--tserver_master_addrs=127.19.144.126:46629
--webserver_port=46021
--webserver_interface=127.19.144.66
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.66.keytab
--principal=oryx/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.144.126:46629,127.19.144.125:35749
--tserver_master_addrs=127.19.144.126:46629,127.19.144.125:35749,127.19.144.124:36719 with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:39:15.697837 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.680190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:38771 (local address 127.19.144.126:46629)
0808 14:39:15.680325 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.680329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.681774 (+ 1445us) server_negotiation.cc:407] Connection header received
0808 14:39:15.682962 (+ 1188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.682966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.683027 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.683118 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.686659 (+ 3541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.687566 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.688962 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.689386 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.689483 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.693404 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.693407 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.693409 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.693439 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.695492 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.696184 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.696188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.696189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.696285 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.696656 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.696662 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.696665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.696915 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.697020 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.697414 (+ 394us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.697559 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 14:39:15.698179 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.682334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:46577 (local address 127.19.144.124:36719)
0808 14:39:15.682584 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.682600 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.682617 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:39:15.682918 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.682921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.682981 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.683498 (+ 517us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.685410 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.686473 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.687714 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.688115 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.688958 (+ 843us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.691823 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.691829 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.691835 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.691872 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.694458 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.695242 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.695249 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.695254 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.695358 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.695847 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.695853 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.695857 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.696150 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.697804 (+ 1654us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.697888 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.698026 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":81,"threads_started":1}
I20260808 14:39:15.698596 27375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.682623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:35749 (local address 127.19.144.65:36433)
0808 14:39:15.683116 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.683132 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.683293 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.683691 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.683693 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.684078 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.685021 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.685034 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.686288 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.686292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.688354 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.688363 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.689505 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.689511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.689641 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.690614 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.690635 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.692844 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.695629 (+ 2785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.695633 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.695635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.696052 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.696402 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.696405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.696407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.696594 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.697087 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.697090 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.697178 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.697295 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.697416 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":742,"spinlock_wait_cycles":7040,"thread_start_us":86,"threads_started":1}
I20260808 14:39:15.698891 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.680628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.65:38771)
0808 14:39:15.681559 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.681606 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.682691 (+ 1085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.683323 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.683332 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.686313 (+ 2981us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.686488 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.686499 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.687724 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.687730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.688797 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.688812 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.689695 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.689702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.689848 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.690744 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.690764 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.693250 (+ 2486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.695629 (+ 2379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.695633 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.695635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.696052 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.696402 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.696405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.696407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.696533 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.697087 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.697091 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.697198 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.698092 (+ 894us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.698349 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":2,"spinlock_wait_cycles":11392,"thread_start_us":576,"threads_started":1}
I20260808 14:39:15.699398 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.682850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:36433 (local address 127.19.144.125:35749)
0808 14:39:15.682996 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.683000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.683210 (+ 210us) server_negotiation.cc:407] Connection header received
0808 14:39:15.683416 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.683420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.683482 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.683540 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.685191 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.686085 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.688493 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.688839 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.689325 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.693035 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.693042 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.693049 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.693367 (+ 318us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.695492 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.696184 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.696187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.696189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.696285 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.696719 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.696722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.696724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.696955 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.697044 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.699061 (+ 2017us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.699202 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 14:39:15.699712 27373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.682203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:36719 (local address 127.19.144.65:46577)
0808 14:39:15.682533 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.682552 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.682696 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.683323 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.683332 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.684078 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.684825 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.684850 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.686607 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.686610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.687571 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.687582 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.688306 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.688311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.688564 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.689324 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.689347 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.691574 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.694613 (+ 3039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.694624 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.694648 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.695093 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.695472 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.695478 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.695483 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.695699 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.696311 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.696321 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.696732 (+ 411us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.696959 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.697374 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":23,"thread_start_us":109,"threads_started":1}
I20260808 14:39:15.700695 27368 heartbeater.cc:347] Connected to a master server at 127.19.144.126:46629
I20260808 14:39:15.700738 27369 heartbeater.cc:347] Connected to a master server at 127.19.144.125:35749
I20260808 14:39:15.700794 27367 heartbeater.cc:347] Connected to a master server at 127.19.144.124:36719
I20260808 14:39:15.700985 27369 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:15.700985 27367 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:15.700987 27368 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:15.701725 27367 heartbeater.cc:511] Master 127.19.144.124:36719 requested a full tablet report, sending...
I20260808 14:39:15.701726 27368 heartbeater.cc:511] Master 127.19.144.126:46629 requested a full tablet report, sending...
I20260808 14:39:15.701725 27369 heartbeater.cc:511] Master 127.19.144.125:35749 requested a full tablet report, sending...
I20260808 14:39:15.703312 26338 ts_manager.cc:194] Re-registered known tserver with Master: 90167cc5e982452a89df283e189e1d92 (127.19.144.65:43199)
I20260808 14:39:15.703661 27184 ts_manager.cc:194] Registered new tserver with Master: 90167cc5e982452a89df283e189e1d92 (127.19.144.65:43199)
I20260808 14:39:15.704247 26338 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:38771
I20260808 14:39:15.705307 26976 ts_manager.cc:194] Registered new tserver with Master: 90167cc5e982452a89df283e189e1d92 (127.19.144.65:43199)
I20260808 14:39:15.767048 26825 heartbeater.cc:503] Master 127.19.144.126:46629 was elected leader, sending a full tablet report...
W20260808 14:39:15.823136 27376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:15.823488 27376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:15.823621 27376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:15.827934 27376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:15.828009 27376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:15.828109 27376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:39:15.833815 27376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.66.keytab
--principal=oryx/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:41105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=46021
--tserver_master_addrs=127.19.144.126:46629,127.19.144.125:35749,127.19.144.124:36719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:15.834986 27376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:15.836105 27376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:15.847083 27383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:15.847262 27376 server_base.cc:1034] running on GCE node
W20260808 14:39:15.847116 27382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:15.847457 27385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:15.847906 27376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:15.848659 27376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:15.849859 27376 hybrid_clock.cc:648] HybridClock initialized: now 1786199955849836 us; error 38 us; skew 500 ppm
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199955, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:15.853235 27376 init.cc:377] Logged in from keytab as oryx/127.19.144.66@KRBTEST.COM (short username oryx)
I20260808 14:39:15.854321 27376 webserver.cc:460] Webserver started at http://127.19.144.66:46021/ using document root <none> and password file <none>
I20260808 14:39:15.854961 27376 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:15.855089 27376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:15.859470 27376 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 14:39:15.862052 27393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:15.863519 27376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:15.863672 27376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "29ecf3b442034a2bb6a52f6177787f43"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "d7005e0bbbfc0996c28c6b12bd555250"
server_key_iv: "c2775eccd40e0c1b9a0c45ee85196d48"
server_key_version: "encryptionkey@0"
I20260808 14:39:15.864368 27376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:15.875675 27376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:15.879326 27376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:15.879557 27376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:15.880211 27376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:15.881186 27376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:15.881263 27376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:15.881335 27376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:15.881386 27376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:15.890956 27376 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:41105
I20260808 14:39:15.891022 27512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:41105 every 8 connection(s)
I20260808 14:39:15.892256 27376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:39:15.894568 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 27376
I20260808 14:39:15.894767 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 26690
I20260808 14:39:15.905396 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:38887
--local_ip_for_outbound_sockets=127.19.144.67
--tserver_master_addrs=127.19.144.126:46629
--webserver_port=42571
--webserver_interface=127.19.144.67
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.67.keytab
--principal=oryx/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.144.126:46629,127.19.144.125:35749
--tserver_master_addrs=127.19.144.126:46629,127.19.144.125:35749,127.19.144.124:36719 with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199955, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.125@KRBTEST.COM
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199955, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.124@KRBTEST.COM
Aug 08 14:39:15 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199955, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM
I20260808 14:39:15.914346 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.896661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:48445 (local address 127.19.144.124:36719)
0808 14:39:15.896836 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.896843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.897301 (+ 458us) server_negotiation.cc:407] Connection header received
0808 14:39:15.898520 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.898524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.898585 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.898643 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.900973 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.901849 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.904278 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.904652 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.904732 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.909866 (+ 5134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.909873 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.909879 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.909916 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.911993 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.912538 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.912542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.912544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.912627 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.912948 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.912951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.912953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.913185 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.913251 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.914017 (+ 766us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.914165 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 14:39:15.914346 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.897084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:60911 (local address 127.19.144.125:35749)
0808 14:39:15.897212 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.897218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.897950 (+ 732us) server_negotiation.cc:407] Connection header received
0808 14:39:15.898520 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.898524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.898585 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.898643 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.900976 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.901849 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.904278 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.904647 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.904732 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.909396 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.909401 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.909404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.909438 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.911567 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.912192 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.912198 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.912203 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.912303 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.912753 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.912757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.912759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.913047 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.913126 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.914017 (+ 891us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.914164 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 14:39:15.915541 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.894293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:39491 (local address 127.19.144.126:46629)
0808 14:39:15.894487 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:15.894492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:15.897176 (+ 2684us) server_negotiation.cc:407] Connection header received
0808 14:39:15.898518 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:15.898524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:15.898594 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:15.898651 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:15.900965 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.901971 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.904272 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.904701 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.904807 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.909402 (+ 4595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:15.909409 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:15.909414 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:15.909453 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:15.912951 (+ 3498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.913531 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.913537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.913542 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.913680 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:15.914073 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:15.914079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:15.914083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:15.914352 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:15.914441 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:15.915131 (+ 690us) server_negotiation.cc:299] Negotiation successful
0808 14:39:15.915284 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:39:15.916334 27517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.896918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:35749 (local address 127.19.144.66:60911)
0808 14:39:15.897831 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.897847 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.898248 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.899093 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.899103 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.899863 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.900775 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.900795 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.902066 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.902070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.904126 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.904138 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.905090 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.905094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.905359 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.906296 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.906308 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.909178 (+ 2870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.911710 (+ 2532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.911717 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.911737 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.912037 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.912430 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.912432 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.912434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.912638 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.913217 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.913227 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.913686 (+ 459us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.914767 (+ 1081us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.915175 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"mutex_wait_us":953,"spinlock_wait_cycles":8960,"thread_start_us":91,"threads_started":1}
I20260808 14:39:15.916335 27519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.896550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:36719 (local address 127.19.144.66:48445)
0808 14:39:15.897204 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.897219 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.898248 (+ 1029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.899093 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.899103 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.899863 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.900773 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.900795 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.902066 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.902070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.904126 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.904138 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.904982 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.904986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.905330 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.906161 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.906213 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.909643 (+ 3430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.912121 (+ 2478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.912126 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.912130 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.912423 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.912747 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.912750 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.912751 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.912839 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.913320 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.913322 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.913686 (+ 364us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.914621 (+ 935us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.915123 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":807,"spinlock_wait_cycles":85120,"thread_start_us":79,"threads_started":1}
I20260808 14:39:15.916386 27518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:15.894754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.66:39491)
0808 14:39:15.897004 (+ 2250us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:15.897067 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:15.898253 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:15.899093 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:15.899105 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:15.899865 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:15.900771 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.900803 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.902152 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.902167 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.904105 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:15.904120 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:15.905009 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:15.905012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:15.905301 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:15.906453 (+ 1152us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:15.906474 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:15.909211 (+ 2737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:15.913100 (+ 3889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.913105 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.913109 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.913393 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.913834 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:15.913840 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:15.913845 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:15.913950 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:15.914527 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:15.914533 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:15.914902 (+ 369us) client_negotiation.cc:769] Sending connection context
0808 14:39:15.915919 (+ 1017us) client_negotiation.cc:240] Negotiation successful
0808 14:39:15.916113 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1992,"mutex_wait_us":391,"spinlock_wait_cycles":28160,"thread_start_us":768,"threads_started":1}
I20260808 14:39:15.918013 27513 heartbeater.cc:347] Connected to a master server at 127.19.144.124:36719
I20260808 14:39:15.918089 27514 heartbeater.cc:347] Connected to a master server at 127.19.144.126:46629
I20260808 14:39:15.918318 27514 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:15.918321 27513 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:15.918037 27515 heartbeater.cc:347] Connected to a master server at 127.19.144.125:35749
I20260808 14:39:15.918457 27515 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:15.919013 27514 heartbeater.cc:511] Master 127.19.144.126:46629 requested a full tablet report, sending...
I20260808 14:39:15.919018 27515 heartbeater.cc:511] Master 127.19.144.125:35749 requested a full tablet report, sending...
I20260808 14:39:15.919664 27513 heartbeater.cc:511] Master 127.19.144.124:36719 requested a full tablet report, sending...
I20260808 14:39:15.920428 27184 ts_manager.cc:194] Registered new tserver with Master: 29ecf3b442034a2bb6a52f6177787f43 (127.19.144.66:41105)
I20260808 14:39:15.920444 26976 ts_manager.cc:194] Registered new tserver with Master: 29ecf3b442034a2bb6a52f6177787f43 (127.19.144.66:41105)
I20260808 14:39:15.920864 26338 ts_manager.cc:194] Re-registered known tserver with Master: 29ecf3b442034a2bb6a52f6177787f43 (127.19.144.66:41105)
I20260808 14:39:15.921540 26338 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:39491
W20260808 14:39:16.031884 27520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:16.032244 27520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:16.032325 27520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:16.036499 27520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:16.036605 27520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:16.036700 27520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:39:16.042548 27520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.67.keytab
--principal=oryx/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:38887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=42571
--tserver_master_addrs=127.19.144.126:46629,127.19.144.125:35749,127.19.144.124:36719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:16.043701 27520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:16.044821 27520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:16.055203 27526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:16.055315 27525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:16.055538 27520 server_base.cc:1034] running on GCE node
W20260808 14:39:16.055675 27528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:16.056110 27520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:16.056969 27520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:16.058147 27520 hybrid_clock.cc:648] HybridClock initialized: now 1786199956058119 us; error 48 us; skew 500 ppm
I20260808 14:39:16.059515 27530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.054970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:36719 (local address 127.0.0.1:41152)
0808 14:39:16.055906 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:16.055919 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:16.056017 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:16.056321 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:16.056324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:16.056349 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:16.056462 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.056468 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.057425 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.057429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.058388 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.058399 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.059085 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.059089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.059179 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.059203 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:39:16.059287 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 14:39:16.059369 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"thread_start_us":86,"threads_started":1}
I20260808 14:39:16.059506 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.055036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41152 (local address 127.19.144.124:36719)
0808 14:39:16.055198 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.055206 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.055999 (+ 793us) server_negotiation.cc:407] Connection header received
0808 14:39:16.056114 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.056117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.056192 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.056303 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:16.056574 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.057304 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.058517 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.058950 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.059085 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.059233 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.059301 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.059367 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 14:39:16.060102 27196 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 14:39:16.060411 27225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0828373b803c4ef5a62d163e24b66108. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } } }
I20260808 14:39:16.060570 27225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: This master's current role is: LEARNER
Aug 08 14:39:16 dist-test-slave-p3t5 krb5kdc[26291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199956, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:16.061492 27520 init.cc:377] Logged in from keytab as oryx/127.19.144.67@KRBTEST.COM (short username oryx)
I20260808 14:39:16.062474 27520 webserver.cc:460] Webserver started at http://127.19.144.67:42571/ using document root <none> and password file <none>
I20260808 14:39:16.063089 27520 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:16.063237 27520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:16.067390 27520 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:39:16.069779 27536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:16.071098 27520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:39:16.071254 27520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7"
format_stamp: "Formatted at 2026-08-08 14:39:14 on dist-test-slave-p3t5"
server_key: "7424b0259e6997b97afe73d5ef7eed8a"
server_key_iv: "583d0144c27e1cafabce1e9a4aa4d52b"
server_key_version: "encryptionkey@0"
I20260808 14:39:16.071858 27520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:16.102339 27520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:16.106226 27520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:16.106467 27520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:16.107137 27520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:16.108105 27520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:16.108183 27520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:16.108260 27520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:16.108319 27520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:16.118155 27520 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:38887
I20260808 14:39:16.118196 27654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:38887 every 8 connection(s)
I20260808 14:39:16.119226 27520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:39:16.124634 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 27520
Aug 08 14:39:16 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199956, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.125@KRBTEST.COM
Aug 08 14:39:16 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199956, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM
Aug 08 14:39:16 dist-test-slave-p3t5 krb5kdc[26291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199956, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.124@KRBTEST.COM
I20260808 14:39:16.139894 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.123330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:49963 (local address 127.19.144.125:35749)
0808 14:39:16.123474 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.123478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.123599 (+ 121us) server_negotiation.cc:407] Connection header received
0808 14:39:16.124251 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.124256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.124318 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.124379 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:16.126400 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.127191 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.129094 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.129447 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.129552 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.133894 (+ 4342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:16.133898 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:16.133903 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:16.133937 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:16.136005 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.137099 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.137103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.137105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.137200 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.137714 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.137719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.137724 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.137969 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:16.138066 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.139569 (+ 1503us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.139706 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60,"spinlock_wait_cycles":2432}
I20260808 14:39:16.141000 27659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.121744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.67:36465)
0808 14:39:16.122710 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:16.122757 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:16.123861 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:16.124529 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:16.124540 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:16.125255 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:16.126208 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.126230 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.127574 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.127577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.129864 (+ 2287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.129873 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.130610 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.130615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.130748 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.131251 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:16.131266 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:16.133686 (+ 2420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:16.136422 (+ 2736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:16.136425 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:16.136427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:16.136767 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:16.137640 (+ 873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:16.137647 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:16.137651 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:16.137749 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:16.139375 (+ 1626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:16.139378 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:16.139514 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 14:39:16.139619 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 14:39:16.139764 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"mutex_wait_us":154,"spinlock_wait_cycles":17920,"thread_start_us":566,"threads_started":1}
I20260808 14:39:16.141594 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.123201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:35749 (local address 127.19.144.67:49963)
0808 14:39:16.123499 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:16.123516 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:16.123867 (+ 351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:16.124531 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:16.124541 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:16.125254 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:16.126208 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.126230 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.127367 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.127372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.128967 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.128981 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.129747 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.129751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.130016 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.130757 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:16.130781 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:16.133686 (+ 2905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:16.136214 (+ 2528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:16.136221 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:16.136241 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:16.136970 (+ 729us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:16.137318 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:16.137321 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:16.137323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:16.137542 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:16.138128 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:16.138136 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:16.138554 (+ 418us) client_negotiation.cc:769] Sending connection context
0808 14:39:16.138835 (+ 281us) client_negotiation.cc:240] Negotiation successful
0808 14:39:16.139271 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":217,"spinlock_wait_cycles":12928,"thread_start_us":112,"threads_started":1}
I20260808 14:39:16.142079 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.121243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:36465 (local address 127.19.144.126:46629)
0808 14:39:16.121428 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.121440 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.122876 (+ 1436us) server_negotiation.cc:407] Connection header received
0808 14:39:16.124091 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.124096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.124161 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.124244 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:16.126390 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.127167 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.130007 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.130359 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.130439 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.133917 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:16.133921 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:16.133924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:16.133962 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:16.136310 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.136876 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.136883 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.136885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.137000 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.138924 (+ 1924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.138928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.138940 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.139213 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:16.139321 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.141804 (+ 2483us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.141926 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 14:39:16.142072 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.127340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48126 (local address 127.19.144.125:35749)
0808 14:39:16.127634 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.127639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.128028 (+ 389us) server_negotiation.cc:407] Connection header received
0808 14:39:16.128212 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.128215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.128269 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.128332 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:16.129477 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.130322 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.132125 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.132501 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.132576 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.137911 (+ 5335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:16.137916 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:16.137920 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:16.137953 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:16.139988 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.140586 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.140589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.140591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.140679 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.140989 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.140996 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.140997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.141226 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:16.141334 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.141769 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.141910 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":105,"threads_started":1}
I20260808 14:39:16.143198 27657 heartbeater.cc:347] Connected to a master server at 127.19.144.125:35749
I20260808 14:39:16.143465 27657 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:16.144101 27657 heartbeater.cc:511] Master 127.19.144.125:35749 requested a full tablet report, sending...
I20260808 14:39:16.144343 27661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.123606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:36719 (local address 127.19.144.67:51051)
0808 14:39:16.123858 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:16.123873 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:16.123963 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:16.124634 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:16.124636 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:16.125255 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:16.127763 (+ 2508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.127775 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.128979 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.128984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.132315 (+ 3331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.132331 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.133926 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.133930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.134044 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.134623 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:16.134643 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:16.136232 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:16.142239 (+ 6007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:16.142244 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:16.142249 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:16.142507 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:16.142825 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:16.142830 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:16.142832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:16.142927 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:16.143493 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:16.143497 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:16.143761 (+ 264us) client_negotiation.cc:769] Sending connection context
0808 14:39:16.143873 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 14:39:16.144040 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":186,"thread_start_us":93,"threads_started":1}
I20260808 14:39:16.144582 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.123834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:51051 (local address 127.19.144.124:36719)
0808 14:39:16.123960 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.123964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.123981 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:39:16.124391 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.124395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.124446 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.124524 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:16.127962 (+ 3438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.128596 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.132482 (+ 3886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.132867 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.132959 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.136400 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:16.136405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:16.136409 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:16.136435 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:16.138774 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.142635 (+ 3861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.142638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.142640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.142714 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.143048 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.143054 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.143059 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.143318 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:16.143426 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.144253 (+ 827us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.144380 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:39:16.144728 27656 heartbeater.cc:347] Connected to a master server at 127.19.144.126:46629
I20260808 14:39:16.144824 27656 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:16.145004 27656 heartbeater.cc:511] Master 127.19.144.126:46629 requested a full tablet report, sending...
I20260808 14:39:16.145941 26976 ts_manager.cc:194] Registered new tserver with Master: bd3bf41b5d504ecba9db5ca6ac52cef7 (127.19.144.67:38887)
I20260808 14:39:16.145990 26338 ts_manager.cc:194] Re-registered known tserver with Master: bd3bf41b5d504ecba9db5ca6ac52cef7 (127.19.144.67:38887)
I20260808 14:39:16.146093 27655 heartbeater.cc:347] Connected to a master server at 127.19.144.124:36719
I20260808 14:39:16.146176 27655 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:16.146342 27655 heartbeater.cc:511] Master 127.19.144.124:36719 requested a full tablet report, sending...
I20260808 14:39:16.146786 26338 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:36465
I20260808 14:39:16.147074 27668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.127363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41356 (local address 127.19.144.126:46629)
0808 14:39:16.129409 (+ 2046us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.129415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.129430 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:39:16.129469 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.129472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.129520 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.130476 (+ 956us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:16.130603 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.131439 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.132554 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.132874 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.133728 (+ 854us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.136787 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:16.136793 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:16.136799 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:16.136834 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:16.144647 (+ 7813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.145147 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.145151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.145153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.145247 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.145557 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.145560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.145562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.146002 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:16.146672 (+ 670us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.146768 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.146905 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1950,"thread_start_us":107,"threads_started":1}
I20260808 14:39:16.147212 27184 ts_manager.cc:194] Registered new tserver with Master: bd3bf41b5d504ecba9db5ca6ac52cef7 (127.19.144.67:38887)
I20260808 14:39:16.150688 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.128730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41158 (local address 127.19.144.124:36719)
0808 14:39:16.128985 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.128989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.134639 (+ 5650us) server_negotiation.cc:407] Connection header received
0808 14:39:16.134687 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.134690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.134750 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.134857 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:16.135797 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.136594 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.137758 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.138113 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.138207 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.139793 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:16.139797 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:16.139800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:16.139824 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:16.141704 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.149009 (+ 7305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.149013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.149015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.149114 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.149432 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.149436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.149438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.149708 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:16.149843 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.150302 (+ 459us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.150507 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":114,"threads_started":1}
I20260808 14:39:16.151262 26338 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:39:16.154619 26338 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:39:16.157451 27144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: NON_VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: true } } }
I20260808 14:39:16.157582 27144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:16.159797 27675 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108: attempting to promote NON_VOTER 67e5b6d0c46b4db088c2ef22d8bdd22b to VOTER
I20260808 14:39:16.160202 27675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } }
I20260808 14:39:16.161439 27196 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 LEARNER]: Refusing update from remote peer 0828373b803c4ef5a62d163e24b66108: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 14:39:16.161463 26982 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 FOLLOWER]: Refusing update from remote peer 0828373b803c4ef5a62d163e24b66108: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 14:39:16.161876 27673 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 14:39:16.162034 27144 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 14:39:16.163923 27675 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 67e5b6d0c46b4db088c2ef22d8bdd22b (127.19.144.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } } }
I20260808 14:39:16.164443 27196 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 67e5b6d0c46b4db088c2ef22d8bdd22b (127.19.144.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } } }
I20260808 14:39:16.164568 26982 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 67e5b6d0c46b4db088c2ef22d8bdd22b (127.19.144.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } } }
I20260808 14:39:16.165678 27673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } } }
I20260808 14:39:16.165781 27673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:16.167706 27225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } } }
I20260808 14:39:16.167831 27225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:39:16.168339 27673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } } }
I20260808 14:39:16.168368 27675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } } }
I20260808 14:39:16.168478 27675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:16.168462 27673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:16.169189 27684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0828373b803c4ef5a62d163e24b66108" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } } }
I20260808 14:39:16.169325 27684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:39:16.179916 27686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.174337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:38887 (local address 127.0.0.1:46694)
0808 14:39:16.175847 (+ 1510us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:16.175860 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:16.175975 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:16.176490 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:16.176493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:16.176512 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:16.176609 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.176615 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.177550 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.177555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.178673 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.178682 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.179518 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.179521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.179604 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.179629 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 14:39:16.179692 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 14:39:16.179765 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1443,"thread_start_us":94,"threads_started":1}
I20260808 14:39:16.179976 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.173880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46694 (local address 127.19.144.67:38887)
0808 14:39:16.174444 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.174458 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.175969 (+ 1511us) server_negotiation.cc:407] Connection header received
0808 14:39:16.176168 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.176172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.176324 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.176518 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:16.176706 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.177430 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.178798 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.179384 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.179521 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.179686 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.179738 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.179831 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":148,"threads_started":1}
I20260808 14:39:16.183210 27588 tablet_service.cc:1467] Processing CreateTablet for tablet 31e4277a2bd943e985a058147bc177ca (DEFAULT_TABLE table=test-table [id=adc3666244414e209c923a94f4098b96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:16.184530 27588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31e4277a2bd943e985a058147bc177ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:16.185534 27689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.173321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:41105 (local address 127.0.0.1:49204)
0808 14:39:16.174764 (+ 1443us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:16.174779 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:16.174864 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:16.175590 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:16.175593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:16.175612 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:16.175719 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.175726 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.182838 (+ 7112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.182842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.183945 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.183958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.185074 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.185078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.185181 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.185210 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 14:39:16.185300 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 14:39:16.185380 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1380,"thread_start_us":123,"threads_started":1}
I20260808 14:39:16.185606 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.173498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49204 (local address 127.19.144.66:41105)
0808 14:39:16.175076 (+ 1578us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.175090 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.175115 (+ 25us) server_negotiation.cc:407] Connection header received
0808 14:39:16.175199 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.175203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.175439 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.175612 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:16.175862 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.182707 (+ 6845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.184112 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.184938 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.185059 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.185211 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.185347 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.185456 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1121,"thread_start_us":121,"threads_started":1}
I20260808 14:39:16.189087 27446 tablet_service.cc:1467] Processing CreateTablet for tablet 31e4277a2bd943e985a058147bc177ca (DEFAULT_TABLE table=test-table [id=adc3666244414e209c923a94f4098b96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:16.190313 27446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31e4277a2bd943e985a058147bc177ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:16.191417 27530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.172287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:43199 (local address 127.0.0.1:45542)
0808 14:39:16.172416 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:16.172432 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:16.172515 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:16.173661 (+ 1146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:16.173664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:16.173686 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:16.173798 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.173806 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.175326 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.175329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.176410 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.176421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.190851 (+ 14430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.190856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.190991 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.191034 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 14:39:16.191119 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 14:39:16.191236 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 14:39:16.191840 27685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.172708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45542 (local address 127.19.144.65:43199)
0808 14:39:16.173033 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.173052 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.173074 (+ 22us) server_negotiation.cc:407] Connection header received
0808 14:39:16.173181 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.173187 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.173449 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.173684 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:16.173903 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.175205 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.189724 (+ 14519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.190666 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.190815 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.190980 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.191523 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.191633 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":124,"threads_started":1}
I20260808 14:39:16.193661 27691 tablet_bootstrap.cc:492] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7: Bootstrap starting.
I20260808 14:39:16.195068 27299 tablet_service.cc:1467] Processing CreateTablet for tablet 31e4277a2bd943e985a058147bc177ca (DEFAULT_TABLE table=test-table [id=adc3666244414e209c923a94f4098b96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:16.195660 27691 tablet_bootstrap.cc:654] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:16.196233 27299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31e4277a2bd943e985a058147bc177ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:16.196451 27691 log.cc:824] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:16.198554 27691 tablet_bootstrap.cc:492] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7: No bootstrap required, opened a new log
I20260808 14:39:16.198777 27691 ts_tablet_manager.cc:1397] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 14:39:16.199070 27693 tablet_bootstrap.cc:492] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43: Bootstrap starting.
I20260808 14:39:16.202075 27693 tablet_bootstrap.cc:654] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:16.203168 27693 log.cc:824] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:16.203893 27691 raft_consensus.cc:348] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:16.204227 27691 raft_consensus.cc:374] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:16.204289 27691 raft_consensus.cc:729] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd3bf41b5d504ecba9db5ca6ac52cef7, State: Initialized, Role: FOLLOWER
I20260808 14:39:16.204764 27691 consensus_queue.cc:260] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:16.205479 27695 tablet_bootstrap.cc:492] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92: Bootstrap starting.
I20260808 14:39:16.205545 27693 tablet_bootstrap.cc:492] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43: No bootstrap required, opened a new log
I20260808 14:39:16.205771 27693 ts_tablet_manager.cc:1397] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 14:39:16.205874 27691 ts_tablet_manager.cc:1428] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260808 14:39:16.206215 27656 heartbeater.cc:503] Master 127.19.144.126:46629 was elected leader, sending a full tablet report...
I20260808 14:39:16.210209 27695 tablet_bootstrap.cc:654] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:16.211015 27695 log.cc:824] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:16.211014 27693 raft_consensus.cc:348] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:16.211325 27693 raft_consensus.cc:374] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:16.211414 27693 raft_consensus.cc:729] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29ecf3b442034a2bb6a52f6177787f43, State: Initialized, Role: FOLLOWER
I20260808 14:39:16.211905 27693 consensus_queue.cc:260] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:16.212651 27514 heartbeater.cc:503] Master 127.19.144.126:46629 was elected leader, sending a full tablet report...
I20260808 14:39:16.212927 27693 ts_tablet_manager.cc:1428] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 14:39:16.213457 27695 tablet_bootstrap.cc:492] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92: No bootstrap required, opened a new log
I20260808 14:39:16.213688 27695 ts_tablet_manager.cc:1397] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 14:39:16.217702 27695 raft_consensus.cc:348] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:16.217981 27695 raft_consensus.cc:374] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:16.218030 27695 raft_consensus.cc:729] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90167cc5e982452a89df283e189e1d92, State: Initialized, Role: FOLLOWER
I20260808 14:39:16.218487 27695 consensus_queue.cc:260] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:16.219184 27368 heartbeater.cc:503] Master 127.19.144.126:46629 was elected leader, sending a full tablet report...
I20260808 14:39:16.219651 27695 ts_tablet_manager.cc:1428] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 14:39:16.299115 27699 raft_consensus.cc:482] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:39:16.299391 27699 raft_consensus.cc:504] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:16.300573 27699 leader_election.cc:288] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29ecf3b442034a2bb6a52f6177787f43 (127.19.144.66:41105), bd3bf41b5d504ecba9db5ca6ac52cef7 (127.19.144.67:38887)
I20260808 14:39:16.305228 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.301512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:35825 (local address 127.19.144.67:38887)
0808 14:39:16.301670 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.301676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.301717 (+ 41us) server_negotiation.cc:407] Connection header received
0808 14:39:16.301908 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.301910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.301977 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.302066 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:16.302333 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.302822 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.304129 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.304646 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.304773 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.304830 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.305017 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.305092 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:39:16.305251 27701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.301333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:38887 (local address 127.19.144.65:35825)
0808 14:39:16.301605 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:16.301642 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:16.301770 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:16.302079 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:16.302081 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:16.302098 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:16.302222 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.302227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.302932 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.302935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.303998 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.304009 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.304766 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.304770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.304875 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.304899 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:39:16.305000 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 14:39:16.305078 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":30,"thread_start_us":106,"threads_started":1}
I20260808 14:39:16.305241 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.300977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:37579 (local address 127.19.144.66:41105)
0808 14:39:16.301120 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.301124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.301315 (+ 191us) server_negotiation.cc:407] Connection header received
0808 14:39:16.301513 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.301517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.301578 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.301668 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:16.302154 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.302692 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.304135 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.304664 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.304799 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.304853 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.305043 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.305121 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:39:16.305260 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.300862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:41105 (local address 127.19.144.65:37579)
0808 14:39:16.301192 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:16.301210 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:16.301347 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:16.301826 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:16.301829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:16.301924 (+ 95us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:16.302035 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.302041 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.302823 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.302827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.304004 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:16.304014 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.304799 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.304803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:16.304903 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.304935 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 14:39:16.305023 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 14:39:16.305097 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":85,"thread_start_us":150,"threads_started":1}
I20260808 14:39:16.306239 27467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31e4277a2bd943e985a058147bc177ca" candidate_uuid: "90167cc5e982452a89df283e189e1d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29ecf3b442034a2bb6a52f6177787f43" is_pre_election: true
I20260808 14:39:16.306615 27467 raft_consensus.cc:2454] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90167cc5e982452a89df283e189e1d92 in term 0.
I20260808 14:39:16.307190 27252 leader_election.cc:302] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29ecf3b442034a2bb6a52f6177787f43, 90167cc5e982452a89df283e189e1d92; no voters:
I20260808 14:39:16.307197 27608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31e4277a2bd943e985a058147bc177ca" candidate_uuid: "90167cc5e982452a89df283e189e1d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" is_pre_election: true
I20260808 14:39:16.307490 27608 raft_consensus.cc:2454] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90167cc5e982452a89df283e189e1d92 in term 0.
I20260808 14:39:16.307550 27699 raft_consensus.cc:2781] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:39:16.307628 27699 raft_consensus.cc:482] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:39:16.307684 27699 raft_consensus.cc:3037] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:16.308915 27699 raft_consensus.cc:504] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:16.309291 27699 leader_election.cc:288] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [CANDIDATE]: Term 1 election: Requested vote from peers 29ecf3b442034a2bb6a52f6177787f43 (127.19.144.66:41105), bd3bf41b5d504ecba9db5ca6ac52cef7 (127.19.144.67:38887)
I20260808 14:39:16.309772 27608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31e4277a2bd943e985a058147bc177ca" candidate_uuid: "90167cc5e982452a89df283e189e1d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7"
I20260808 14:39:16.309772 27467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31e4277a2bd943e985a058147bc177ca" candidate_uuid: "90167cc5e982452a89df283e189e1d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29ecf3b442034a2bb6a52f6177787f43"
I20260808 14:39:16.309911 27608 raft_consensus.cc:3037] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:16.309911 27467 raft_consensus.cc:3037] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:16.311313 27608 raft_consensus.cc:2454] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90167cc5e982452a89df283e189e1d92 in term 1.
I20260808 14:39:16.311314 27467 raft_consensus.cc:2454] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90167cc5e982452a89df283e189e1d92 in term 1.
I20260808 14:39:16.311612 27252 leader_election.cc:302] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29ecf3b442034a2bb6a52f6177787f43, 90167cc5e982452a89df283e189e1d92; no voters:
I20260808 14:39:16.311833 27699 raft_consensus.cc:2781] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:16.312081 27699 raft_consensus.cc:686] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [term 1 LEADER]: Becoming Leader. State: Replica: 90167cc5e982452a89df283e189e1d92, State: Running, Role: LEADER
I20260808 14:39:16.312470 27699 consensus_queue.cc:237] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:16.316869 26338 catalog_manager.cc:5495] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90167cc5e982452a89df283e189e1d92 (127.19.144.65). New cstate: current_term: 1 leader_uuid: "90167cc5e982452a89df283e189e1d92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } health_report { overall_health: UNKNOWN } } }
I20260808 14:39:16.330767 27685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.326984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45546 (local address 127.19.144.65:43199)
0808 14:39:16.327112 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.327116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.327131 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:39:16.327270 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.327273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.327329 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.327448 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:39:16.327724 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.328266 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.329173 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.329397 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.329511 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.329885 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:39:16.330339 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:39:16.330483 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.330547 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.330613 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 14:39:16.337249 27608 raft_consensus.cc:1261] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 1 FOLLOWER]: Refusing update from remote peer 90167cc5e982452a89df283e189e1d92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:16.337249 27467 raft_consensus.cc:1261] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43 [term 1 FOLLOWER]: Refusing update from remote peer 90167cc5e982452a89df283e189e1d92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:16.338130 27699 consensus_queue.cc:1035] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:16.338366 27702 consensus_queue.cc:1035] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:16.345352 27705 mvcc.cc:205] Tried to move back new op lower bound from 7316275021149483008 to 7316275021059203072. Current Snapshot: MvccSnapshot[applied={T|T < 7316275021149483008}]
I20260808 14:39:16.348840 27706 mvcc.cc:205] Tried to move back new op lower bound from 7316275021149483008 to 7316275021059203072. Current Snapshot: MvccSnapshot[applied={T|T < 7316275021149483008}]
I20260808 14:39:16.353608 27707 mvcc.cc:205] Tried to move back new op lower bound from 7316275021149483008 to 7316275021059203072. Current Snapshot: MvccSnapshot[applied={T|T < 7316275021149483008 or (T in {7316275021149483008})}]
I20260808 14:39:16.360350 26336 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:16.360561 26336 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:16.365587 26336 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0828373b803c4ef5a62d163e24b66108: Sending DeleteTablet for 3 replicas of tablet 31e4277a2bd943e985a058147bc177ca
I20260808 14:39:16.366503 27299 tablet_service.cc:1514] Processing DeleteTablet for tablet 31e4277a2bd943e985a058147bc177ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:16 UTC) from {username='oryx', principal='oryx/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:45542
I20260808 14:39:16.366817 27446 tablet_service.cc:1514] Processing DeleteTablet for tablet 31e4277a2bd943e985a058147bc177ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:16 UTC) from {username='oryx', principal='oryx/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:49204
I20260808 14:39:16.367002 27588 tablet_service.cc:1514] Processing DeleteTablet for tablet 31e4277a2bd943e985a058147bc177ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:16 UTC) from {username='oryx', principal='oryx/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:46694
I20260808 14:39:16.367329 27715 tablet_replica.cc:330] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43: stopping tablet replica
I20260808 14:39:16.367717 27715 raft_consensus.cc:2229] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:16.368098 27715 raft_consensus.cc:2258] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:16.369709 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 26305
I20260808 14:39:16.369820 27714 tablet_replica.cc:330] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92: stopping tablet replica
I20260808 14:39:16.374223 27714 raft_consensus.cc:2229] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:39:16.374728 27714 raft_consensus.cc:2258] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:16.375506 27715 ts_tablet_manager.cc:1905] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:16.377925 27714 ts_tablet_manager.cc:1905] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:16.378587 27715 ts_tablet_manager.cc:1918] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:16.378701 27715 log.cc:1196] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/31e4277a2bd943e985a058147bc177ca
I20260808 14:39:16.379060 27715 ts_tablet_manager.cc:1939] T 31e4277a2bd943e985a058147bc177ca P 29ecf3b442034a2bb6a52f6177787f43: Deleting consensus metadata
I20260808 14:39:16.380872 27714 ts_tablet_manager.cc:1918] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:16.380995 27714 log.cc:1196] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-0/wal/wals/31e4277a2bd943e985a058147bc177ca
I20260808 14:39:16.381438 27714 ts_tablet_manager.cc:1939] T 31e4277a2bd943e985a058147bc177ca P 90167cc5e982452a89df283e189e1d92: Deleting consensus metadata
I20260808 14:39:16.381850 27716 tablet_replica.cc:330] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7: stopping tablet replica
I20260808 14:39:16.382200 27716 raft_consensus.cc:2229] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:16.382622 27716 raft_consensus.cc:2258] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:16.384197 27716 ts_tablet_manager.cc:1905] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 14:39:16.386940 27396 connection.cc:535] server connection from 127.0.0.1:49204 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 14:39:16.387104 27250 connection.cc:535] server connection from 127.0.0.1:45542 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 14:39:16.387303 27716 ts_tablet_manager.cc:1918] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:16.387404 27716 log.cc:1196] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/31e4277a2bd943e985a058147bc177ca
I20260808 14:39:16.387626 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.368691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48128 (local address 127.19.144.125:35749)
0808 14:39:16.368803 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.368808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.375431 (+ 6623us) server_negotiation.cc:407] Connection header received
0808 14:39:16.376177 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.376184 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.376269 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.376340 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:16.377528 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.378330 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.379900 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.380249 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.380339 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.382510 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:16.382514 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:16.382516 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:16.382550 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:16.384578 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.385132 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.385138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.385143 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.385253 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.385596 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.385602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.385606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.385930 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:16.386090 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.386402 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.386707 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 14:39:16.387784 27716 ts_tablet_manager.cc:1939] T 31e4277a2bd943e985a058147bc177ca P bd3bf41b5d504ecba9db5ca6ac52cef7: Deleting consensus metadata
W20260808 14:39:16.389113 27537 connection.cc:441] server connection from 127.0.0.1:46694 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46694 (request call id 1) could send its response
I20260808 14:39:16.400795 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.369433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41170 (local address 127.19.144.124:36719)
0808 14:39:16.369543 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.369547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.386781 (+ 17234us) server_negotiation.cc:407] Connection header received
0808 14:39:16.388115 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.388119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.388193 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.388296 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:16.389113 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.389909 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.393377 (+ 3468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.393833 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.393922 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.395900 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:16.395906 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:16.395911 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:16.395947 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:16.397942 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.399060 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.399064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.399066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.399174 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.399542 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.399550 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.399553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.399805 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:16.399921 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.400378 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.400568 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 14:39:16.403615 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.393043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48138 (local address 127.19.144.125:35749)
0808 14:39:16.393199 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.393204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.393220 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:39:16.393267 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.393272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.393328 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.393408 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:16.394978 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.395840 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.397220 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.397588 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.397702 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.399758 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:16.399762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:16.399765 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:16.399808 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:16.401625 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.402172 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.402175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.402179 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.402280 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.402665 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.402669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.402671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.402865 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:16.403011 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.403286 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.403412 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 14:39:16.404603 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:16.391746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41176 (local address 127.19.144.124:36719)
0808 14:39:16.391939 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:16.391945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:16.393592 (+ 1647us) server_negotiation.cc:407] Connection header received
0808 14:39:16.394000 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:16.394004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:16.394055 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:16.394106 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:16.395015 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.395803 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.397068 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:16.397411 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:16.397494 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:16.399672 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:16.399676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:16.399679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:16.399706 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:16.402307 (+ 2601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.402811 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.402815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.402817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.402901 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:16.403218 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:16.403222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:16.403223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:16.403429 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:16.403551 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:16.404320 (+ 769us) server_negotiation.cc:299] Negotiation successful
0808 14:39:16.404446 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260808 14:39:17.349464 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:17.348948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.67:55997)
0808 14:39:17.349265 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:17.349365 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":80,"threads_started":1}
W20260808 14:39:17.349833 27656 heartbeater.cc:644] Failed to heartbeat to 127.19.144.126:46629 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
I20260808 14:39:17.356997 27727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:17.356359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.65:43671)
0808 14:39:17.356743 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:17.356852 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":106,"threads_started":1}
I20260808 14:39:17.357064 27728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:17.356491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.66:49665)
0808 14:39:17.356852 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:17.356957 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":92,"threads_started":1}
W20260808 14:39:17.357491 27368 heartbeater.cc:644] Failed to heartbeat to 127.19.144.126:46629 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
W20260808 14:39:17.357657 27514 heartbeater.cc:644] Failed to heartbeat to 127.19.144.126:46629 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
I20260808 14:39:17.866627 27729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0828373b803c4ef5a62d163e24b66108)
I20260808 14:39:17.866672 27730 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0828373b803c4ef5a62d163e24b66108)
I20260808 14:39:17.866889 27729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } }
I20260808 14:39:17.866899 27730 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } }
I20260808 14:39:17.867951 27729 leader_election.cc:288] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0828373b803c4ef5a62d163e24b66108 (127.19.144.126:46629), 4cab1eb43c6042ceadf9392eb38e6a2b (127.19.144.125:35749)
I20260808 14:39:17.868005 27730 leader_election.cc:288] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0828373b803c4ef5a62d163e24b66108 (127.19.144.126:46629), 67e5b6d0c46b4db088c2ef22d8bdd22b (127.19.144.124:36719)
I20260808 14:39:17.868673 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:17.868116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:41816)
0808 14:39:17.868477 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:17.868566 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":61,"threads_started":1}
I20260808 14:39:17.868673 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:17.868178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:41810)
0808 14:39:17.868475 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:17.868566 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":94,"threads_started":1}
W20260808 14:39:17.869091 27167 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
W20260808 14:39:17.869087 26960 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
I20260808 14:39:17.870043 27733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:17.869791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:41820)
0808 14:39:17.869898 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:17.869956 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 14:39:17.870184 26960 leader_election.cc:334] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0828373b803c4ef5a62d163e24b66108 (127.19.144.126:46629): Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
I20260808 14:39:17.870599 27738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:17.870249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:41824)
0808 14:39:17.870461 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:17.870522 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":92,"threads_started":1}
W20260808 14:39:17.870795 27167 leader_election.cc:334] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0828373b803c4ef5a62d163e24b66108 (127.19.144.126:46629): Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
I20260808 14:39:17.873744 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:17.868891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:35749 (local address 127.0.0.1:48144)
0808 14:39:17.869649 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:17.869664 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:17.869763 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:17.870075 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:17.870077 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:17.870096 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:17.870262 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:17.870269 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:17.871232 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:17.871235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:17.872459 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:17.872470 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:17.873302 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:17.873305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:17.873388 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:17.873416 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 14:39:17.873488 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 14:39:17.873564 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":29}
I20260808 14:39:17.873765 27734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:17.868635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48144 (local address 127.19.144.125:35749)
0808 14:39:17.869173 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:17.869177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:17.869855 (+ 678us) server_negotiation.cc:407] Connection header received
0808 14:39:17.869901 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:17.869907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:17.869949 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:17.870033 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:17.870398 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:17.871122 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:17.872580 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:17.873175 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:17.873279 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:17.873330 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:17.873520 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 14:39:17.873597 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":110,"threads_started":1}
I20260808 14:39:17.873808 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:17.868481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:36719 (local address 127.0.0.1:41190)
0808 14:39:17.868825 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:17.868839 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:17.868965 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:17.869331 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:17.869338 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:17.869416 (+ 78us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:17.869540 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:17.869548 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:17.871020 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:17.871024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:17.872332 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:17.872344 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:17.873265 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:17.873272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:17.873371 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:17.873400 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 14:39:17.873488 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 14:39:17.873564 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":71,"threads_started":1}
I20260808 14:39:17.873780 27735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:17.868846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41190 (local address 127.19.144.124:36719)
0808 14:39:17.869053 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:17.869058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:17.869073 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:39:17.869124 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:17.869127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:17.869182 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:17.869295 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:17.869697 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:17.870910 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:17.872465 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:17.873104 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:17.873242 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:17.873299 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:17.873521 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 14:39:17.873597 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":67,"threads_started":1}
I20260808 14:39:17.874414 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" is_pre_election: true
I20260808 14:39:17.874451 26982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" is_pre_election: true
I20260808 14:39:17.874637 27196 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cab1eb43c6042ceadf9392eb38e6a2b in term 1.
I20260808 14:39:17.874660 26982 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67e5b6d0c46b4db088c2ef22d8bdd22b in term 1.
I20260808 14:39:17.875011 26960 leader_election.cc:302] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4cab1eb43c6042ceadf9392eb38e6a2b, 67e5b6d0c46b4db088c2ef22d8bdd22b; no voters: 0828373b803c4ef5a62d163e24b66108
I20260808 14:39:17.875276 27730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 14:39:17.875336 27730 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0828373b803c4ef5a62d163e24b66108)
I20260808 14:39:17.875375 27730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 1 FOLLOWER]: Advancing to term 2
I20260808 14:39:17.875568 27167 leader_election.cc:302] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4cab1eb43c6042ceadf9392eb38e6a2b, 67e5b6d0c46b4db088c2ef22d8bdd22b; no voters: 0828373b803c4ef5a62d163e24b66108
I20260808 14:39:17.875794 27729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 14:39:17.875875 27729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0828373b803c4ef5a62d163e24b66108)
I20260808 14:39:17.875936 27729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 1 FOLLOWER]: Advancing to term 2
I20260808 14:39:17.876660 27730 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } }
I20260808 14:39:17.877079 27730 leader_election.cc:288] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [CANDIDATE]: Term 2 election: Requested vote from peers 0828373b803c4ef5a62d163e24b66108 (127.19.144.126:46629), 67e5b6d0c46b4db088c2ef22d8bdd22b (127.19.144.124:36719)
I20260808 14:39:17.877206 27729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } }
I20260808 14:39:17.877414 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:17.877192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:41826)
0808 14:39:17.877282 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:17.877342 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 14:39:17.877586 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b"
I20260808 14:39:17.877710 27729 leader_election.cc:288] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [CANDIDATE]: Term 2 election: Requested vote from peers 0828373b803c4ef5a62d163e24b66108 (127.19.144.126:46629), 4cab1eb43c6042ceadf9392eb38e6a2b (127.19.144.125:35749)
I20260808 14:39:17.877790 27196 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4cab1eb43c6042ceadf9392eb38e6a2b in current term 2: Already voted for candidate 67e5b6d0c46b4db088c2ef22d8bdd22b in this term.
I20260808 14:39:17.878187 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:17.877959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:41846)
0808 14:39:17.878059 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:17.878111 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 14:39:17.878291 26982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b"
I20260808 14:39:17.878441 26982 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 67e5b6d0c46b4db088c2ef22d8bdd22b in current term 2: Already voted for candidate 4cab1eb43c6042ceadf9392eb38e6a2b in this term.
I20260808 14:39:17.879019 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:17.878728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:41852)
0808 14:39:17.878866 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:17.878938 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 14:39:17.879159 27167 leader_election.cc:334] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0828373b803c4ef5a62d163e24b66108 (127.19.144.126:46629): Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
I20260808 14:39:17.879251 27167 leader_election.cc:302] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67e5b6d0c46b4db088c2ef22d8bdd22b; no voters: 0828373b803c4ef5a62d163e24b66108, 4cab1eb43c6042ceadf9392eb38e6a2b
I20260808 14:39:17.879395 27729 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 14:39:17.880257 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:17.877893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:41834)
0808 14:39:17.880112 (+ 2219us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:17.880188 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
W20260808 14:39:17.880383 26960 leader_election.cc:334] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0828373b803c4ef5a62d163e24b66108 (127.19.144.126:46629): Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
I20260808 14:39:17.880472 26960 leader_election.cc:302] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4cab1eb43c6042ceadf9392eb38e6a2b; no voters: 0828373b803c4ef5a62d163e24b66108, 67e5b6d0c46b4db088c2ef22d8bdd22b
I20260808 14:39:17.880601 27730 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 14:39:18.351001 27739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:18.350418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.67:50775)
0808 14:39:18.350750 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:18.350858 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":115,"threads_started":1}
I20260808 14:39:18.358647 27741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:18.358223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.66:58177)
0808 14:39:18.358450 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:18.358524 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":64,"threads_started":1}
I20260808 14:39:18.358788 27740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:18.358171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.65:34997)
0808 14:39:18.358611 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:18.358675 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":100,"threads_started":1}
I20260808 14:39:19.352567 27742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.352080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.67:49619)
0808 14:39:19.352352 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.352415 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":104,"threads_started":1}
I20260808 14:39:19.359987 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.359351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.66:48937)
0808 14:39:19.359735 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.359866 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":81,"threads_started":1}
I20260808 14:39:19.360237 27744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.359778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.65:47321)
0808 14:39:19.360049 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.360133 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":77,"threads_started":1}
I20260808 14:39:19.531713 27745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:39:19.531869 27745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } }
I20260808 14:39:19.532413 27745 leader_election.cc:288] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0828373b803c4ef5a62d163e24b66108 (127.19.144.126:46629), 67e5b6d0c46b4db088c2ef22d8bdd22b (127.19.144.124:36719)
I20260808 14:39:19.533119 27746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.532612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:42106)
0808 14:39:19.532922 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.533003 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":76,"threads_started":1}
I20260808 14:39:19.533273 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" is_pre_election: true
I20260808 14:39:19.533478 27196 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cab1eb43c6042ceadf9392eb38e6a2b in term 2.
I20260808 14:39:19.533999 26960 leader_election.cc:302] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cab1eb43c6042ceadf9392eb38e6a2b, 67e5b6d0c46b4db088c2ef22d8bdd22b; no voters:
I20260808 14:39:19.534086 27746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.533852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:42110)
0808 14:39:19.533974 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.534026 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 14:39:19.534188 26960 leader_election.cc:334] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0828373b803c4ef5a62d163e24b66108 (127.19.144.126:46629): Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
I20260808 14:39:19.534251 27745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 14:39:19.534324 27745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:39:19.534380 27745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 2 FOLLOWER]: Advancing to term 3
I20260808 14:39:19.535521 27745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } }
I20260808 14:39:19.535863 27745 leader_election.cc:288] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [CANDIDATE]: Term 3 election: Requested vote from peers 0828373b803c4ef5a62d163e24b66108 (127.19.144.126:46629), 67e5b6d0c46b4db088c2ef22d8bdd22b (127.19.144.124:36719)
I20260808 14:39:19.536216 27746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.536021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:42120)
0808 14:39:19.536120 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.536165 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 14:39:19.536448 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b"
I20260808 14:39:19.536590 27196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 2 FOLLOWER]: Advancing to term 3
I20260808 14:39:19.536959 27746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.536700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:42128)
0808 14:39:19.536833 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.536889 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 14:39:19.537086 26960 leader_election.cc:334] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0828373b803c4ef5a62d163e24b66108 (127.19.144.126:46629): Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
I20260808 14:39:19.537830 27196 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cab1eb43c6042ceadf9392eb38e6a2b in term 3.
I20260808 14:39:19.538152 26960 leader_election.cc:302] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4cab1eb43c6042ceadf9392eb38e6a2b, 67e5b6d0c46b4db088c2ef22d8bdd22b; no voters: 0828373b803c4ef5a62d163e24b66108
I20260808 14:39:19.538314 27745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 3 FOLLOWER]: Leader election won for term 3
I20260808 14:39:19.538484 27745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [term 3 LEADER]: Becoming Leader. State: Replica: 4cab1eb43c6042ceadf9392eb38e6a2b, State: Running, Role: LEADER
I20260808 14:39:19.538755 27745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } }
I20260808 14:39:19.539901 27748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cab1eb43c6042ceadf9392eb38e6a2b. Latest consensus state: current_term: 3 leader_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } } }
I20260808 14:39:19.540055 27748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:19.540441 27750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:19.542876 27750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:19.543295 27750 catalog_manager.cc:1259] Loaded cluster ID: df6391f5877f42caa44d19deb4f08545
I20260808 14:39:19.543365 27750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:19.543931 27750 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:19.544345 27750 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: Loaded TSK: 0
I20260808 14:39:19.545070 27750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:39:19.572121 26974 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:39:19.575541 26974 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:39:19.578859 27746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.578605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:42152)
0808 14:39:19.578726 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.578785 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 14:39:19.579201 27196 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [term 3 FOLLOWER]: Refusing update from remote peer 4cab1eb43c6042ceadf9392eb38e6a2b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 14:39:19.579617 27746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.579378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.0.0.1:42162)
0808 14:39:19.579480 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.579545 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 14:39:19.579741 27748 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 14:39:19.579993 26960 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b -> Peer 0828373b803c4ef5a62d163e24b66108 (127.19.144.126:46629): Couldn't send request to peer 0828373b803c4ef5a62d163e24b66108. Status: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 14:39:19.581360 27752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cab1eb43c6042ceadf9392eb38e6a2b. Latest consensus state: current_term: 3 leader_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } } }
I20260808 14:39:19.581509 27752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:39:19.582676 27745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } } }
I20260808 14:39:19.582844 27745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:19.583250 27752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0828373b803c4ef5a62d163e24b66108" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46629 } } peers { permanent_uuid: "4cab1eb43c6042ceadf9392eb38e6a2b" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 35749 } attrs { promote: false } } peers { permanent_uuid: "67e5b6d0c46b4db088c2ef22d8bdd22b" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 36719 } attrs { promote: false } } }
I20260808 14:39:19.583367 27752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e5b6d0c46b4db088c2ef22d8bdd22b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:39:19.595044 27746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.588972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:41105 (local address 127.0.0.1:49212)
0808 14:39:19.589173 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.589188 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:19.589316 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:19.590901 (+ 1585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:19.590903 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:19.590937 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:19.591167 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:19.591174 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:19.592334 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:19.592338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:19.593689 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:19.593701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:19.594612 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:19.594616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:19.594725 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:19.594754 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 14:39:19.594806 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 14:39:19.594877 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":93,"spinlock_wait_cycles":5120}
I20260808 14:39:19.595862 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.589935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49212 (local address 127.19.144.66:41105)
0808 14:39:19.590669 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:19.590673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:19.590687 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:39:19.590738 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:19.590741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:19.590788 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:19.590886 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:19.591320 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:19.592205 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:19.593833 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:19.594472 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:19.595518 (+ 1046us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:19.595592 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:19.595633 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 14:39:19.595714 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":108,"threads_started":1}
I20260808 14:39:19.596776 27446 tablet_service.cc:1467] Processing CreateTablet for tablet 357f524f2e6e47b38379b584335df651 (DEFAULT_TABLE table=test-table [id=0b5c63cb286c4f7f9802a52ee7ad1eb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:19.597095 27446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 357f524f2e6e47b38379b584335df651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:19.598205 27757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.589486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:43199 (local address 127.0.0.1:45560)
0808 14:39:19.589780 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.589794 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:19.589882 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:19.594092 (+ 4210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:19.594095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:19.594113 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:19.594208 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:19.594214 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:19.595283 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:19.595289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:19.596364 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:19.596376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:19.597760 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:19.597764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:19.597852 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:19.597876 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:39:19.597991 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 14:39:19.598068 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":158,"threads_started":1}
I20260808 14:39:19.598209 27761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.590029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45560 (local address 127.19.144.65:43199)
0808 14:39:19.593833 (+ 3804us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:19.593838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:19.593852 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:39:19.593904 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:19.593907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:19.593956 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:19.594313 (+ 357us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:19.594424 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:19.595141 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:19.596680 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:19.597608 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:19.597762 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:19.597824 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:19.598013 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 14:39:19.598090 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3696,"thread_start_us":104,"threads_started":1}
I20260808 14:39:19.598819 27299 tablet_service.cc:1467] Processing CreateTablet for tablet 357f524f2e6e47b38379b584335df651 (DEFAULT_TABLE table=test-table [id=0b5c63cb286c4f7f9802a52ee7ad1eb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:19.599072 27299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 357f524f2e6e47b38379b584335df651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:19.599879 27762 tablet_bootstrap.cc:492] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43: Bootstrap starting.
I20260808 14:39:19.600375 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.589935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46700 (local address 127.19.144.67:38887)
0808 14:39:19.590224 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:19.590229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:19.590454 (+ 225us) server_negotiation.cc:407] Connection header received
0808 14:39:19.590553 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:19.590557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:19.590615 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:19.590703 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:19.591330 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:19.592196 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:19.595146 (+ 2950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:19.595823 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:19.595901 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:19.595962 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:19.596398 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 14:39:19.596484 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":99,"threads_started":1}
I20260808 14:39:19.600894 27762 tablet_bootstrap.cc:654] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:19.600910 27758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.589504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:38887 (local address 127.0.0.1:46700)
0808 14:39:19.590357 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.590370 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:19.590463 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:19.590895 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:19.590899 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:19.590929 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:19.591166 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:19.591174 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:19.592338 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:19.592341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:19.594985 (+ 2644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:19.594999 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:19.596159 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:19.596162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:19.596267 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:19.596289 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 14:39:19.600605 (+ 4316us) client_negotiation.cc:240] Negotiation successful
0808 14:39:19.600710 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"mutex_wait_us":1505,"spinlock_wait_cycles":4736,"thread_start_us":142,"threads_started":1}
I20260808 14:39:19.601547 27763 tablet_bootstrap.cc:492] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92: Bootstrap starting.
I20260808 14:39:19.601575 27588 tablet_service.cc:1467] Processing CreateTablet for tablet 357f524f2e6e47b38379b584335df651 (DEFAULT_TABLE table=test-table [id=0b5c63cb286c4f7f9802a52ee7ad1eb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:19.601922 27588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 357f524f2e6e47b38379b584335df651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:19.602445 27762 tablet_bootstrap.cc:492] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43: No bootstrap required, opened a new log
I20260808 14:39:19.602542 27762 ts_tablet_manager.cc:1397] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:39:19.602794 27763 tablet_bootstrap.cc:654] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:19.603099 27762 raft_consensus.cc:348] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:19.603224 27762 raft_consensus.cc:374] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:19.603260 27762 raft_consensus.cc:729] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29ecf3b442034a2bb6a52f6177787f43, State: Initialized, Role: FOLLOWER
I20260808 14:39:19.603403 27762 consensus_queue.cc:260] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:19.603833 27762 ts_tablet_manager.cc:1428] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:39:19.604215 27763 tablet_bootstrap.cc:492] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92: No bootstrap required, opened a new log
I20260808 14:39:19.604318 27763 ts_tablet_manager.cc:1397] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:39:19.604485 27768 tablet_bootstrap.cc:492] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7: Bootstrap starting.
I20260808 14:39:19.604714 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.604320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.66:49285)
0808 14:39:19.604582 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.604640 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
I20260808 14:39:19.604804 27763 raft_consensus.cc:348] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:19.604933 27763 raft_consensus.cc:374] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:19.604962 27763 raft_consensus.cc:729] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90167cc5e982452a89df283e189e1d92, State: Initialized, Role: FOLLOWER
I20260808 14:39:19.605070 27763 consensus_queue.cc:260] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:19.605382 27768 tablet_bootstrap.cc:654] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:19.605453 27763 ts_tablet_manager.cc:1428] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:39:19.606220 27744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.605944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.65:39015)
0808 14:39:19.606086 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.606148 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 14:39:19.608083 27744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.607725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.65:37481)
0808 14:39:19.607944 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.607997 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260808 14:39:19.608331 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.608080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.66:45469)
0808 14:39:19.608198 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.608256 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 14:39:19.608903 27369 heartbeater.cc:503] Master 127.19.144.125:35749 was elected leader, sending a full tablet report...
I20260808 14:39:19.609376 27768 tablet_bootstrap.cc:492] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7: No bootstrap required, opened a new log
I20260808 14:39:19.609506 27768 ts_tablet_manager.cc:1397] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260808 14:39:19.609977 27768 raft_consensus.cc:348] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:19.610095 27768 raft_consensus.cc:374] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:19.610117 27768 raft_consensus.cc:729] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd3bf41b5d504ecba9db5ca6ac52cef7, State: Initialized, Role: FOLLOWER
I20260808 14:39:19.610252 27768 consensus_queue.cc:260] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:19.610663 27768 ts_tablet_manager.cc:1428] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:39:19.611284 27742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.611055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.67:42009)
0808 14:39:19.611172 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.611224 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 14:39:19.611984 27742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.611746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.67:41361)
0808 14:39:19.611868 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.611923 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 14:39:19.612569 27657 heartbeater.cc:503] Master 127.19.144.125:35749 was elected leader, sending a full tablet report...
W20260808 14:39:19.624238 27658 tablet.cc:2369] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:39:19.646689 27516 tablet.cc:2369] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:39:19.669049 27769 raft_consensus.cc:482] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:39:19.669255 27769 raft_consensus.cc:504] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:19.669809 27769 leader_election.cc:288] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29ecf3b442034a2bb6a52f6177787f43 (127.19.144.66:41105), bd3bf41b5d504ecba9db5ca6ac52cef7 (127.19.144.67:38887)
I20260808 14:39:19.670267 27467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "357f524f2e6e47b38379b584335df651" candidate_uuid: "90167cc5e982452a89df283e189e1d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29ecf3b442034a2bb6a52f6177787f43" is_pre_election: true
I20260808 14:39:19.670364 27608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "357f524f2e6e47b38379b584335df651" candidate_uuid: "90167cc5e982452a89df283e189e1d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" is_pre_election: true
I20260808 14:39:19.670455 27467 raft_consensus.cc:2454] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90167cc5e982452a89df283e189e1d92 in term 0.
I20260808 14:39:19.670559 27608 raft_consensus.cc:2454] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90167cc5e982452a89df283e189e1d92 in term 0.
I20260808 14:39:19.670802 27252 leader_election.cc:302] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29ecf3b442034a2bb6a52f6177787f43, 90167cc5e982452a89df283e189e1d92; no voters:
I20260808 14:39:19.671032 27769 raft_consensus.cc:2781] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:39:19.671109 27769 raft_consensus.cc:482] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:39:19.671162 27769 raft_consensus.cc:3037] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:19.672222 27769 raft_consensus.cc:504] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:19.672575 27769 leader_election.cc:288] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [CANDIDATE]: Term 1 election: Requested vote from peers 29ecf3b442034a2bb6a52f6177787f43 (127.19.144.66:41105), bd3bf41b5d504ecba9db5ca6ac52cef7 (127.19.144.67:38887)
I20260808 14:39:19.673019 27608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "357f524f2e6e47b38379b584335df651" candidate_uuid: "90167cc5e982452a89df283e189e1d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7"
I20260808 14:39:19.673019 27467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "357f524f2e6e47b38379b584335df651" candidate_uuid: "90167cc5e982452a89df283e189e1d92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29ecf3b442034a2bb6a52f6177787f43"
I20260808 14:39:19.673148 27467 raft_consensus.cc:3037] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:19.673148 27608 raft_consensus.cc:3037] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:19.674336 27467 raft_consensus.cc:2454] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90167cc5e982452a89df283e189e1d92 in term 1.
I20260808 14:39:19.674336 27608 raft_consensus.cc:2454] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90167cc5e982452a89df283e189e1d92 in term 1.
I20260808 14:39:19.674656 27252 leader_election.cc:302] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90167cc5e982452a89df283e189e1d92, bd3bf41b5d504ecba9db5ca6ac52cef7; no voters:
I20260808 14:39:19.674834 27769 raft_consensus.cc:2781] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:19.675071 27769 raft_consensus.cc:686] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [term 1 LEADER]: Becoming Leader. State: Replica: 90167cc5e982452a89df283e189e1d92, State: Running, Role: LEADER
I20260808 14:39:19.675238 27769 consensus_queue.cc:237] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } } peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } }
I20260808 14:39:19.675841 27744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.675563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.65:34567)
0808 14:39:19.675692 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.675754 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 14:39:19.677093 26974 catalog_manager.cc:5495] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90167cc5e982452a89df283e189e1d92 (127.19.144.65). New cstate: current_term: 1 leader_uuid: "90167cc5e982452a89df283e189e1d92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90167cc5e982452a89df283e189e1d92" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 } health_report { overall_health: UNKNOWN } } }
W20260808 14:39:19.682662 27370 tablet.cc:2369] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:39:19.696720 27761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.693834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45568 (local address 127.19.144.65:43199)
0808 14:39:19.693980 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:19.693983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:19.693998 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:39:19.694106 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:19.694108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:19.694153 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:19.694217 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:39:19.694463 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:19.694970 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:19.695770 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:19.695979 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:19.696055 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:19.696277 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:39:19.696349 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:39:19.696401 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:19.696512 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 14:39:19.696561 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 14:39:19.698982 27608 raft_consensus.cc:1261] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 1 FOLLOWER]: Refusing update from remote peer 90167cc5e982452a89df283e189e1d92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:19.699016 27467 raft_consensus.cc:1261] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43 [term 1 FOLLOWER]: Refusing update from remote peer 90167cc5e982452a89df283e189e1d92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:19.699510 27769 consensus_queue.cc:1035] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29ecf3b442034a2bb6a52f6177787f43" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:19.699643 27772 consensus_queue.cc:1035] T 357f524f2e6e47b38379b584335df651 P 90167cc5e982452a89df283e189e1d92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd3bf41b5d504ecba9db5ca6ac52cef7" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:19.700701 27515 heartbeater.cc:503] Master 127.19.144.125:35749 was elected leader, sending a full tablet report...
I20260808 14:39:19.701751 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.701163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.66:38381)
0808 14:39:19.701566 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.701669 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328}
I20260808 14:39:19.702869 27742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.702174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.67:36969)
0808 14:39:19.702728 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.702787 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483}
I20260808 14:39:19.704720 27744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.704291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.65:34835)
0808 14:39:19.704585 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.704643 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221}
I20260808 14:39:19.708083 27742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.706065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.67:57819)
0808 14:39:19.707958 (+ 1893us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.708012 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1821}
I20260808 14:39:19.708915 26974 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:19.709153 26974 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:19.709504 27744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.708760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.65:35855)
0808 14:39:19.709379 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.709436 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":550}
I20260808 14:39:19.709856 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.709318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46629 (local address 127.19.144.66:44595)
0808 14:39:19.709709 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.709774 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:46629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320}
I20260808 14:39:19.714449 26974 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4cab1eb43c6042ceadf9392eb38e6a2b: Sending DeleteTablet for 3 replicas of tablet 357f524f2e6e47b38379b584335df651
I20260808 14:39:19.715216 27588 tablet_service.cc:1514] Processing DeleteTablet for tablet 357f524f2e6e47b38379b584335df651 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:19 UTC) from {username='oryx', principal='oryx/127.19.144.125@KRBTEST.COM'} at 127.0.0.1:46700
I20260808 14:39:19.715245 27299 tablet_service.cc:1514] Processing DeleteTablet for tablet 357f524f2e6e47b38379b584335df651 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:19 UTC) from {username='oryx', principal='oryx/127.19.144.125@KRBTEST.COM'} at 127.0.0.1:45560
I20260808 14:39:19.715392 27298 tablet_service.cc:1514] Processing DeleteTablet for tablet 357f524f2e6e47b38379b584335df651 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:19 UTC) from {username='oryx', principal='oryx/127.19.144.125@KRBTEST.COM'} at 127.0.0.1:45560
I20260808 14:39:19.715675 27785 tablet_replica.cc:330] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7: stopping tablet replica
I20260808 14:39:19.715854 27446 tablet_service.cc:1514] Processing DeleteTablet for tablet 357f524f2e6e47b38379b584335df651 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:19 UTC) from {username='oryx', principal='oryx/127.19.144.125@KRBTEST.COM'} at 127.0.0.1:49212
I20260808 14:39:19.715888 27785 raft_consensus.cc:2229] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:19.716053 27785 raft_consensus.cc:2258] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:19.716195 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 27133
I20260808 14:39:19.716228 27787 tablet_replica.cc:330] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43: stopping tablet replica
I20260808 14:39:19.717235 27787 raft_consensus.cc:2229] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:19.717427 27787 raft_consensus.cc:2258] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:19.718446 27787 ts_tablet_manager.cc:1905] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:19.720158 27785 ts_tablet_manager.cc:1905] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:19.721141 27787 ts_tablet_manager.cc:1918] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:19.721233 27787 log.cc:1196] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/357f524f2e6e47b38379b584335df651
I20260808 14:39:19.721661 27787 ts_tablet_manager.cc:1939] T 357f524f2e6e47b38379b584335df651 P 29ecf3b442034a2bb6a52f6177787f43: Deleting consensus metadata
I20260808 14:39:19.722525 26960 catalog_manager.cc:4841] TS 29ecf3b442034a2bb6a52f6177787f43 (127.19.144.66:41105): tablet 357f524f2e6e47b38379b584335df651 (table test-table [id=0b5c63cb286c4f7f9802a52ee7ad1eb9]) successfully deleted
I20260808 14:39:19.722959 27785 ts_tablet_manager.cc:1918] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:19.723064 27785 log.cc:1196] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/357f524f2e6e47b38379b584335df651
I20260808 14:39:19.723315 27785 ts_tablet_manager.cc:1939] T 357f524f2e6e47b38379b584335df651 P bd3bf41b5d504ecba9db5ca6ac52cef7: Deleting consensus metadata
I20260808 14:39:19.724030 26960 catalog_manager.cc:4841] TS bd3bf41b5d504ecba9db5ca6ac52cef7 (127.19.144.67:38887): tablet 357f524f2e6e47b38379b584335df651 (table test-table [id=0b5c63cb286c4f7f9802a52ee7ad1eb9]) successfully deleted
I20260808 14:39:19.727425 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 27376
I20260808 14:39:19.727627 27758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:19.727298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:43199 (local address 127.0.0.1:45582)
0808 14:39:19.727439 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:19.727496 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:43199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 14:39:19.728528 26962 catalog_manager.cc:4584] TS 90167cc5e982452a89df283e189e1d92 (127.19.144.65:43199): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 357f524f2e6e47b38379b584335df651: Network error: Client connection negotiation failed: client connection to 127.19.144.65:43199: connect: Connection refused (error 111)
I20260808 14:39:19.737331 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 27520
I20260808 14:39:19.744920 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 26830
I20260808 14:39:19.753229 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 26935
2026-08-08T14:39:19Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7216 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:39:19 dist-test-slave-p3t5 krb5kdc[27790](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:39:19 dist-test-slave-p3t5 krb5kdc[27790](info): set up 2 sockets
Aug 08 14:39:19 dist-test-slave-p3t5 krb5kdc[27790](info): commencing operation
krb5kdc: starting...
W20260808 14:39:20.824965 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:39:20 dist-test-slave-p3t5 krb5kdc[27790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:39:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:20Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.126.keytab.
Entry for principal oryx/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.126.keytab.
I20260808 14:39:20.962656 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:43957
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.126.keytab
--principal=oryx/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:44729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:43957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:21.080432 27803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:21.080752 27803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:21.080844 27803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:21.084965 27803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:21.085069 27803 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 14:39:21.085101 27803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:21.085144 27803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:21.085186 27803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:21.085235 27803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:39:21.090940 27803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:44729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:43957
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.126.keytab
--principal=oryx/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:43957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:21.092172 27803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:21.093289 27803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:21.102401 27808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:21.102415 27811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:21.102406 27809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:21.102587 27803 server_base.cc:1034] running on GCE node
I20260808 14:39:21.103271 27803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:21.104452 27803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:21.105688 27803 hybrid_clock.cc:648] HybridClock initialized: now 1786199961105674 us; error 27 us; skew 500 ppm
Aug 08 14:39:21 dist-test-slave-p3t5 krb5kdc[27790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199961, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:21.108599 27803 init.cc:377] Logged in from keytab as oryx/127.19.144.126@KRBTEST.COM (short username oryx)
I20260808 14:39:21.109503 27803 webserver.cc:460] Webserver started at http://127.19.144.126:41611/ using document root <none> and password file <none>
I20260808 14:39:21.110112 27803 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:21.110177 27803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:21.110348 27803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:21.112406 27803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "8a265713a6614240ae6de7eb9f6bcf3d"
format_stamp: "Formatted at 2026-08-08 14:39:21 on dist-test-slave-p3t5"
server_key: "20645ce2a09d053fa73d6b6f1c14f2eb"
server_key_iv: "27fb4e215ca745b37cdf04b50c3992f8"
server_key_version: "encryptionkey@0"
I20260808 14:39:21.112941 27803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "8a265713a6614240ae6de7eb9f6bcf3d"
format_stamp: "Formatted at 2026-08-08 14:39:21 on dist-test-slave-p3t5"
server_key: "20645ce2a09d053fa73d6b6f1c14f2eb"
server_key_iv: "27fb4e215ca745b37cdf04b50c3992f8"
server_key_version: "encryptionkey@0"
I20260808 14:39:21.116883 27803 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:39:21.119633 27819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:21.121014 27803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:39:21.121160 27803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "8a265713a6614240ae6de7eb9f6bcf3d"
format_stamp: "Formatted at 2026-08-08 14:39:21 on dist-test-slave-p3t5"
server_key: "20645ce2a09d053fa73d6b6f1c14f2eb"
server_key_iv: "27fb4e215ca745b37cdf04b50c3992f8"
server_key_version: "encryptionkey@0"
I20260808 14:39:21.121284 27803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:21.136675 27803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:21.139969 27803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:21.140202 27803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:21.148624 27803 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:43957
I20260808 14:39:21.148643 27882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:43957 every 8 connection(s)
I20260808 14:39:21.149664 27803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:21.150229 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 27803
I20260808 14:39:21.150388 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:39:21.150717 20033 external_mini_cluster.cc:1428] Setting key 0a4e76c88ab72f158d174145363ed8c1
I20260808 14:39:21.153090 27883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 14:39:21 dist-test-slave-p3t5 krb5kdc[27790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM
I20260808 14:39:21.160161 27883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d: Bootstrap starting.
I20260808 14:39:21.162827 27883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:21.163934 27883 log.cc:824] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:21.165782 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:21.152133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40162 (local address 127.19.144.126:43957)
0808 14:39:21.152563 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:21.152580 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:21.152616 (+ 36us) server_negotiation.cc:407] Connection header received
0808 14:39:21.153312 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:21.153331 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:21.153740 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:21.154033 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:21.154991 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.156019 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.157203 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.157769 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.157888 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:21.160853 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:21.160858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:21.160873 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:21.160901 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:21.162700 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:21.163209 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:21.163214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:21.163219 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:21.163337 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:21.163641 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:21.163645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:21.163647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:21.164088 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:21.164316 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:21.164635 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 14:39:21.164980 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":142,"threads_started":1}
I20260808 14:39:21.166083 27883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d: No bootstrap required, opened a new log
I20260808 14:39:21.169435 27883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a265713a6614240ae6de7eb9f6bcf3d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43957 } }
I20260808 14:39:21.169730 27883 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:21.169790 27883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a265713a6614240ae6de7eb9f6bcf3d, State: Initialized, Role: FOLLOWER
I20260808 14:39:21.170162 27883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a265713a6614240ae6de7eb9f6bcf3d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43957 } }
I20260808 14:39:21.170281 27883 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:21.170327 27883 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:21.170397 27883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:21.171248 27883 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a265713a6614240ae6de7eb9f6bcf3d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43957 } }
I20260808 14:39:21.171538 27883 leader_election.cc:302] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a265713a6614240ae6de7eb9f6bcf3d; no voters:
I20260808 14:39:21.171806 27883 leader_election.cc:288] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:39:21.171959 27889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:21.172212 27889 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [term 1 LEADER]: Becoming Leader. State: Replica: 8a265713a6614240ae6de7eb9f6bcf3d, State: Running, Role: LEADER
I20260808 14:39:21.172554 27889 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a265713a6614240ae6de7eb9f6bcf3d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43957 } }
I20260808 14:39:21.173192 27883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:21.174566 27891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a265713a6614240ae6de7eb9f6bcf3d. Latest consensus state: current_term: 1 leader_uuid: "8a265713a6614240ae6de7eb9f6bcf3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a265713a6614240ae6de7eb9f6bcf3d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43957 } } }
I20260808 14:39:21.174576 27890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a265713a6614240ae6de7eb9f6bcf3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a265713a6614240ae6de7eb9f6bcf3d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43957 } } }
I20260808 14:39:21.174728 27890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:21.175012 27891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:21.175544 27900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:21.178853 27900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:21.184741 27900 catalog_manager.cc:1347] Generated new cluster ID: 96f3fc3c002e4294aea18ec6ea6cfb03
I20260808 14:39:21.184836 27900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:21.194581 27900 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:39:21.195631 27900 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:21.207993 27900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d: Generated new TSK 0
I20260808 14:39:21.208745 27900 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.65.keytab.
Entry for principal oryx/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.65.keytab.
I20260808 14:39:21.245092 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:43957
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.65.keytab
--principal=oryx/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:44729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:21.369093 27911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:21.369423 27911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:21.369475 27911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:21.373799 27911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:21.373903 27911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:21.373960 27911 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:39:21.373994 27911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:39:21.379753 27911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:44729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.65.keytab
--principal=oryx/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.144.126:43957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:21.380884 27911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:21.382057 27911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:21.392805 27919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:21.393000 27911 server_base.cc:1034] running on GCE node
W20260808 14:39:21.392920 27917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:21.392850 27916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:21.393718 27911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:21.394430 27911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:21.395608 27911 hybrid_clock.cc:648] HybridClock initialized: now 1786199961395586 us; error 33 us; skew 500 ppm
Aug 08 14:39:21 dist-test-slave-p3t5 krb5kdc[27790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199961, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:21.398671 27911 init.cc:377] Logged in from keytab as oryx/127.19.144.65@KRBTEST.COM (short username oryx)
I20260808 14:39:21.399654 27911 webserver.cc:460] Webserver started at http://127.19.144.65:40877/ using document root <none> and password file <none>
I20260808 14:39:21.400224 27911 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:21.400297 27911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:21.400476 27911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:21.402704 27911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "3572036761bb4f29b786328b3dcb7588"
format_stamp: "Formatted at 2026-08-08 14:39:21 on dist-test-slave-p3t5"
server_key: "e434bb1a6a6dfb5855c5c913511bf00c"
server_key_iv: "5e944c5dda25c407ca1b481acdaa98d7"
server_key_version: "encryptionkey@0"
I20260808 14:39:21.403264 27911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "3572036761bb4f29b786328b3dcb7588"
format_stamp: "Formatted at 2026-08-08 14:39:21 on dist-test-slave-p3t5"
server_key: "e434bb1a6a6dfb5855c5c913511bf00c"
server_key_iv: "5e944c5dda25c407ca1b481acdaa98d7"
server_key_version: "encryptionkey@0"
I20260808 14:39:21.407205 27911 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:39:21.409832 27926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:21.411149 27911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:39:21.411291 27911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "3572036761bb4f29b786328b3dcb7588"
format_stamp: "Formatted at 2026-08-08 14:39:21 on dist-test-slave-p3t5"
server_key: "e434bb1a6a6dfb5855c5c913511bf00c"
server_key_iv: "5e944c5dda25c407ca1b481acdaa98d7"
server_key_version: "encryptionkey@0"
I20260808 14:39:21.411413 27911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:21.425908 27911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:21.428881 27911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:21.429096 27911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:21.430737 27911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:21.430819 27911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:21.430917 27911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:21.430958 27911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 14:39:21 dist-test-slave-p3t5 krb5kdc[27790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199961, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM
I20260808 14:39:21.442371 27911 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:37131
I20260808 14:39:21.442482 28047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:37131 every 8 connection(s)
I20260808 14:39:21.443636 27911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:39:21.444509 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 27911
I20260808 14:39:21.444645 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:39:21.445007 20033 external_mini_cluster.cc:1428] Setting key ce1e91304047d1727fefe3397b31da26
I20260808 14:39:21.446062 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:21.431501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:52557 (local address 127.19.144.126:43957)
0808 14:39:21.431638 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:21.431641 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:21.432779 (+ 1138us) server_negotiation.cc:407] Connection header received
0808 14:39:21.433830 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:21.433836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:21.433911 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:21.434031 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:21.435451 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.436169 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.437793 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.438091 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.438217 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:21.441585 (+ 3368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:21.441588 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:21.441591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:21.441659 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:21.443436 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:21.444056 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:21.444060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:21.444062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:21.444182 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:21.444680 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:21.444685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:21.444687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:21.444932 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:21.445020 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:21.445748 (+ 728us) server_negotiation.cc:299] Negotiation successful
0808 14:39:21.445888 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 14:39:21.447104 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:21.432060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43957 (local address 127.19.144.65:52557)
0808 14:39:21.432591 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:21.432635 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:21.433524 (+ 889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:21.434450 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:21.434457 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:21.434802 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:21.435285 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:21.435300 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.436302 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.436306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:21.437592 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:21.437603 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.438277 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.438281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:21.438507 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:21.439327 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:21.439344 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:21.441373 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:21.443578 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:21.443583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:21.443596 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:21.443937 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:21.444307 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:21.444312 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:21.444314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:21.444511 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:21.445053 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:21.445060 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:21.445388 (+ 328us) client_negotiation.cc:769] Sending connection context
0808 14:39:21.445606 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 14:39:21.445991 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":144,"threads_started":1}
I20260808 14:39:21.448922 28048 heartbeater.cc:347] Connected to a master server at 127.19.144.126:43957
I20260808 14:39:21.449096 28048 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:21.449702 28048 heartbeater.cc:511] Master 127.19.144.126:43957 requested a full tablet report, sending...
I20260808 14:39:21.451390 27836 ts_manager.cc:194] Registered new tserver with Master: 3572036761bb4f29b786328b3dcb7588 (127.19.144.65:37131)
I20260808 14:39:21.452741 27836 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:52557
WARNING: no policy specified for oryx/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:39:21.459645 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:21.451431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:36951 (local address 127.19.144.126:43957)
0808 14:39:21.451569 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:21.451575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:21.451856 (+ 281us) server_negotiation.cc:407] Connection header received
0808 14:39:21.451972 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:21.451975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:21.452019 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:21.452104 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:21.452711 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.453399 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.454629 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.454946 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.455022 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:21.456394 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:21.456398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:21.456401 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:21.456426 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:21.457957 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:21.458379 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:21.458383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:21.458385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:21.458476 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:21.458802 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:21.458807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:21.458808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:21.458995 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:21.459104 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:21.459365 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 14:39:21.459488 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 14:39:21.459656 28056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:21.451315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43957 (local address 127.19.144.65:36951)
0808 14:39:21.451748 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:21.451765 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:21.451876 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:21.452143 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:21.452148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:21.452414 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:21.452574 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:21.452585 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.453527 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.453530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:21.454486 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:21.454498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.455178 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.455181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:21.455274 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:21.455881 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:21.455893 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:21.456256 (+ 363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:21.458067 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:21.458071 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:21.458073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:21.458277 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:21.458615 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:21.458618 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:21.458620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:21.458707 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:21.459095 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:21.459098 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:21.459200 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 14:39:21.459308 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 14:39:21.459452 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":5,"thread_start_us":126,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.144.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.66.keytab.
Entry for principal oryx/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.66.keytab.
I20260808 14:39:21.478791 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:43957
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.66.keytab
--principal=oryx/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:44729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:21.621490 28058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:21.621877 28058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:21.621935 28058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:21.626137 28058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:21.626238 28058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:21.626314 28058 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:39:21.626380 28058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:39:21.631880 28058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:44729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.66.keytab
--principal=oryx/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.144.126:43957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:21.633019 28058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:21.634095 28058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:21.644286 28063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:21.644330 28066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:21.644331 28064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:21.644547 28058 server_base.cc:1034] running on GCE node
I20260808 14:39:21.645277 28058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:21.646068 28058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:21.647276 28058 hybrid_clock.cc:648] HybridClock initialized: now 1786199961647227 us; error 51 us; skew 500 ppm
Aug 08 14:39:21 dist-test-slave-p3t5 krb5kdc[27790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199961, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:21.650377 28058 init.cc:377] Logged in from keytab as oryx/127.19.144.66@KRBTEST.COM (short username oryx)
I20260808 14:39:21.651317 28058 webserver.cc:460] Webserver started at http://127.19.144.66:41685/ using document root <none> and password file <none>
I20260808 14:39:21.651935 28058 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:21.652037 28058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:21.652297 28058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:21.654565 28058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "3dab0325fbae4f069c67092426e7cc73"
format_stamp: "Formatted at 2026-08-08 14:39:21 on dist-test-slave-p3t5"
server_key: "62a277074368784b9011df11e49d10c7"
server_key_iv: "9b5605babb984284eedd676444d0bb29"
server_key_version: "encryptionkey@0"
I20260808 14:39:21.655174 28058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "3dab0325fbae4f069c67092426e7cc73"
format_stamp: "Formatted at 2026-08-08 14:39:21 on dist-test-slave-p3t5"
server_key: "62a277074368784b9011df11e49d10c7"
server_key_iv: "9b5605babb984284eedd676444d0bb29"
server_key_version: "encryptionkey@0"
I20260808 14:39:21.659159 28058 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:39:21.661794 28073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:21.663028 28058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:39:21.663177 28058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "3dab0325fbae4f069c67092426e7cc73"
format_stamp: "Formatted at 2026-08-08 14:39:21 on dist-test-slave-p3t5"
server_key: "62a277074368784b9011df11e49d10c7"
server_key_iv: "9b5605babb984284eedd676444d0bb29"
server_key_version: "encryptionkey@0"
I20260808 14:39:21.663316 28058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:21.686237 28058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:21.689373 28058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:21.689603 28058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:21.691233 28058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:21.691325 28058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:21.691390 28058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:21.691443 28058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 14:39:21 dist-test-slave-p3t5 krb5kdc[27790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199961, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM
I20260808 14:39:21.703011 28058 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:38523
I20260808 14:39:21.703085 28192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:38523 every 8 connection(s)
I20260808 14:39:21.704209 28058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:39:21.706786 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:21.692244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:42945 (local address 127.19.144.126:43957)
0808 14:39:21.692383 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:21.692387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:21.693598 (+ 1211us) server_negotiation.cc:407] Connection header received
0808 14:39:21.694880 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:21.694887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:21.694962 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:21.695063 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:21.696904 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.697841 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.699064 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.699305 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.699433 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:21.702721 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:21.702725 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:21.702728 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:21.702785 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:21.704396 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:21.705030 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:21.705034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:21.705039 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:21.705148 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:21.705529 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:21.705532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:21.705534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:21.705807 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:21.705923 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:21.706446 (+ 523us) server_negotiation.cc:299] Negotiation successful
0808 14:39:21.706586 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:39:21.707718 28083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:21.692909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43957 (local address 127.19.144.66:42945)
0808 14:39:21.693435 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:21.693484 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:21.694529 (+ 1045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:21.695322 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:21.695344 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:21.695882 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:21.696680 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:21.696700 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.697977 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.697980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:21.698918 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:21.698928 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.699504 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.699509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:21.699683 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:21.700873 (+ 1190us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:21.700891 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:21.702513 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:21.704524 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:21.704530 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:21.704547 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:21.704902 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:21.705252 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:21.705258 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:21.705261 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:21.705412 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:21.705921 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:21.705929 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:21.706183 (+ 254us) client_negotiation.cc:769] Sending connection context
0808 14:39:21.706389 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 14:39:21.706752 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":133,"threads_started":1}
I20260808 14:39:21.707877 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 28058
I20260808 14:39:21.708021 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:39:21.708379 20033 external_mini_cluster.cc:1428] Setting key 48885d2d69425261ba3bf53bceb73aed
I20260808 14:39:21.709169 28193 heartbeater.cc:347] Connected to a master server at 127.19.144.126:43957
I20260808 14:39:21.709332 28193 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:21.709965 28193 heartbeater.cc:511] Master 127.19.144.126:43957 requested a full tablet report, sending...
I20260808 14:39:21.711243 27835 ts_manager.cc:194] Registered new tserver with Master: 3dab0325fbae4f069c67092426e7cc73 (127.19.144.66:38523)
I20260808 14:39:21.712126 27835 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:42945
WARNING: no policy specified for oryx/127.19.144.67@KRBTEST.COM; defaulting to no policy
I20260808 14:39:21.720521 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:21.711958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:40557 (local address 127.19.144.126:43957)
0808 14:39:21.712082 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:21.712087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:21.712831 (+ 744us) server_negotiation.cc:407] Connection header received
0808 14:39:21.713015 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:21.713018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:21.713066 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:21.713119 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:21.713815 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.714552 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.715491 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.715725 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.715844 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:21.717061 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:21.717065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:21.717068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:21.717093 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:21.718741 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:21.719198 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:21.719202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:21.719203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:21.719289 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:21.719604 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:21.719609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:21.719610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:21.719844 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:21.719969 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:21.720232 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 14:39:21.720353 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 14:39:21.720546 28199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:21.711836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43957 (local address 127.19.144.66:40557)
0808 14:39:21.712720 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:21.712760 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:21.712903 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:21.713262 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:21.713264 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:21.713515 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:21.713675 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:21.713684 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.714712 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.714714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:21.715345 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:21.715354 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.715851 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.715854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:21.715950 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:21.716556 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:21.716570 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:21.716919 (+ 349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:21.718866 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:21.718869 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:21.718872 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:21.719092 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:21.719386 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:21.719390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:21.719392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:21.719499 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:21.719950 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:21.719954 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:21.720060 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 14:39:21.720172 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 14:39:21.720344 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"thread_start_us":99,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.144.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.67.keytab.
Entry for principal oryx/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.67.keytab.
I20260808 14:39:21.740830 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:43957
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.67.keytab
--principal=oryx/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:44729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:21.865545 28202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:21.865973 28202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:21.866079 28202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:21.870649 28202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:21.870746 28202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:21.870821 28202 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 14:39:21.870913 28202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:39:21.878028 28202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:44729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.67.keytab
--principal=oryx/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.144.126:43957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:21.879585 28202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:21.880952 28202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:21.892501 28208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:21.892462 28210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:21.892642 28202 server_base.cc:1034] running on GCE node
W20260808 14:39:21.892462 28207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:21.893249 28202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:21.894081 28202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:21.895295 28202 hybrid_clock.cc:648] HybridClock initialized: now 1786199961895276 us; error 29 us; skew 500 ppm
Aug 08 14:39:21 dist-test-slave-p3t5 krb5kdc[27790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199961, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:21.898486 28202 init.cc:377] Logged in from keytab as oryx/127.19.144.67@KRBTEST.COM (short username oryx)
I20260808 14:39:21.899475 28202 webserver.cc:460] Webserver started at http://127.19.144.67:36599/ using document root <none> and password file <none>
I20260808 14:39:21.900149 28202 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:21.900218 28202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:21.900466 28202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:21.902794 28202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb"
format_stamp: "Formatted at 2026-08-08 14:39:21 on dist-test-slave-p3t5"
server_key: "8bc3d8c0f1ab12f9f9bf775c1561695c"
server_key_iv: "b3920aa3eeb562c6272f4854fc238eb7"
server_key_version: "encryptionkey@0"
I20260808 14:39:21.903419 28202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb"
format_stamp: "Formatted at 2026-08-08 14:39:21 on dist-test-slave-p3t5"
server_key: "8bc3d8c0f1ab12f9f9bf775c1561695c"
server_key_iv: "b3920aa3eeb562c6272f4854fc238eb7"
server_key_version: "encryptionkey@0"
I20260808 14:39:21.907660 28202 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:39:21.910933 28217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:21.912354 28202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:39:21.912513 28202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb"
format_stamp: "Formatted at 2026-08-08 14:39:21 on dist-test-slave-p3t5"
server_key: "8bc3d8c0f1ab12f9f9bf775c1561695c"
server_key_iv: "b3920aa3eeb562c6272f4854fc238eb7"
server_key_version: "encryptionkey@0"
I20260808 14:39:21.912647 28202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:21.926925 28202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:21.930593 28202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:21.930819 28202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:21.932380 28202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:21.932433 28202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:21.932476 28202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:21.932530 28202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 14:39:21 dist-test-slave-p3t5 krb5kdc[27790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199961, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM
I20260808 14:39:21.944430 28202 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:40499
I20260808 14:39:21.944516 28336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:40499 every 8 connection(s)
I20260808 14:39:21.945554 28202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:39:21.947743 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:21.933456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:53725 (local address 127.19.144.126:43957)
0808 14:39:21.933597 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:21.933600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:21.934745 (+ 1145us) server_negotiation.cc:407] Connection header received
0808 14:39:21.935917 (+ 1172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:21.935921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:21.935990 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:21.936095 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:21.937824 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.938511 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.939775 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.940090 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.940212 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:21.943556 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:21.943560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:21.943562 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:21.943591 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:21.945207 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:21.945864 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:21.945867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:21.945868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:21.946004 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:21.946426 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:21.946429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:21.946431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:21.946756 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:21.946857 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:21.947384 (+ 527us) server_negotiation.cc:299] Negotiation successful
0808 14:39:21.947546 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 14:39:21.948570 28227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:21.934058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43957 (local address 127.19.144.67:53725)
0808 14:39:21.934586 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:21.934631 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:21.935648 (+ 1017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:21.936329 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:21.936339 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:21.936862 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:21.937643 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:21.937665 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.938654 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.938658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:21.939640 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:21.939651 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.940270 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.940274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:21.940523 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:21.941226 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:21.941247 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:21.943326 (+ 2079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:21.945334 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:21.945340 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:21.945355 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:21.945764 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:21.946132 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:21.946137 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:21.946139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:21.946308 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:21.946894 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:21.946899 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:21.947147 (+ 248us) client_negotiation.cc:769] Sending connection context
0808 14:39:21.947308 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 14:39:21.947616 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":148,"threads_started":1}
I20260808 14:39:21.949105 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 28202
I20260808 14:39:21.949273 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:39:21.949680 20033 external_mini_cluster.cc:1428] Setting key a1e9f2eadb8138d3d3955d763f4b4376
I20260808 14:39:21.950141 28337 heartbeater.cc:347] Connected to a master server at 127.19.144.126:43957
I20260808 14:39:21.950323 28337 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:21.950902 28337 heartbeater.cc:511] Master 127.19.144.126:43957 requested a full tablet report, sending...
I20260808 14:39:21.952078 27836 ts_manager.cc:194] Registered new tserver with Master: 43edeb6b22f84c4bb3629e52b7a6b4bb (127.19.144.67:40499)
I20260808 14:39:21.952812 27836 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:53725
I20260808 14:39:21.959775 28343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:21.952622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43957 (local address 127.19.144.67:36617)
0808 14:39:21.952915 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:21.952932 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:21.953034 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:21.953573 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:21.953575 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:21.953834 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:21.953932 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:21.953939 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.954918 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.954920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:21.955486 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:21.955494 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.955956 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.955958 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:21.956029 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:21.956571 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:21.956589 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:21.956909 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:21.958521 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:21.958524 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:21.958525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:21.958720 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:21.958967 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:21.958970 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:21.958971 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:21.959046 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:21.959364 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:21.959366 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:21.959451 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 14:39:21.959530 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 14:39:21.959637 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":111,"threads_started":1}
I20260808 14:39:21.959795 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:21.952685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:36617 (local address 127.19.144.126:43957)
0808 14:39:21.952817 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:21.952822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:21.952999 (+ 177us) server_negotiation.cc:407] Connection header received
0808 14:39:21.953409 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:21.953412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:21.953461 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:21.953544 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:21.954052 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.954788 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.955624 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.955839 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.955949 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:21.957034 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:21.957037 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:21.957039 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:21.957060 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:21.958423 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:21.958802 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:21.958805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:21.958806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:21.958888 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:21.959120 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:21.959122 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:21.959124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:21.959285 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:21.959359 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:21.959590 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 14:39:21.959684 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 14:39:21.963651 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 14:39:21 dist-test-slave-p3t5 krb5kdc[27790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199961, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 14:39:21 dist-test-slave-p3t5 krb5kdc[27790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:21 dist-test-slave-p3t5 krb5kdc[27790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 14:39:21.991704 27887 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:21.984817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40178 (local address 127.19.144.126:43957)
0808 14:39:21.984988 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:21.984992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:21.985006 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:39:21.985056 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:21.985058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:21.985101 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:21.985176 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:21.985984 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.986572 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.987432 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.987679 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.987768 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:21.991492 (+ 3724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40178: BlockingRecv error: recv got EOF from 127.0.0.1:40178 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 08 14:39:21 dist-test-slave-p3t5 krb5kdc[27790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199961, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM
I20260808 14:39:22.001791 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:21.993201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40194 (local address 127.19.144.126:43957)
0808 14:39:21.993356 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:21.993361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:21.993514 (+ 153us) server_negotiation.cc:407] Connection header received
0808 14:39:21.993688 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:21.993691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:21.993738 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:21.993814 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:21.994408 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.994961 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.995740 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:21.995965 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:21.996060 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:21.998530 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:21.998533 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:21.998536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:21.998562 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:22.000060 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:22.000472 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:22.000475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:22.000477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:22.000564 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:22.000864 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:22.000867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:22.000868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:22.001068 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:22.001201 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:22.001394 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 14:39:22.001497 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:39:22.005043 27836 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40194:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:39:22.008677 27836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:39:22.024637 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.018490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:37131 (local address 127.0.0.1:33150)
0808 14:39:22.019709 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:22.019775 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:22.019917 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:22.020505 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:22.020511 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:22.020537 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:22.020705 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.020713 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.021949 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.021953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.023191 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.023202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.024053 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.024056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.024165 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.024209 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 14:39:22.024329 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 14:39:22.024466 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1041,"mutex_wait_us":42,"spinlock_wait_cycles":3712,"thread_start_us":96,"threads_started":1}
I20260808 14:39:22.024637 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.018633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33150 (local address 127.19.144.65:37131)
0808 14:39:22.018907 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:22.018915 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:22.019880 (+ 965us) server_negotiation.cc:407] Connection header received
0808 14:39:22.020091 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:22.020097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:22.020335 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:22.020490 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:22.020856 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.021805 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.023322 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.023929 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.024043 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.024206 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:22.024377 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 14:39:22.024486 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":88,"threads_started":1}
I20260808 14:39:22.025417 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.019266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:40499 (local address 127.0.0.1:44500)
0808 14:39:22.019878 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:22.019893 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:22.020007 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:22.020712 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:22.020715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:22.020733 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:22.020824 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.020833 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.022351 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.022357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.023737 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.023777 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.024980 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.024984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.025070 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.025094 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:39:22.025161 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 14:39:22.025240 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"mutex_wait_us":130,"thread_start_us":96,"threads_started":1}
I20260808 14:39:22.025750 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.019360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34608 (local address 127.19.144.66:38523)
0808 14:39:22.020202 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:22.020212 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:22.020231 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:39:22.020315 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:22.020319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:22.020505 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:22.020667 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:22.021099 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.022243 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.023914 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.024750 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.024872 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.025047 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:22.025494 (+ 447us) server_negotiation.cc:299] Negotiation successful
0808 14:39:22.025591 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":117,"threads_started":1}
I20260808 14:39:22.026036 28360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.018510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:38523 (local address 127.0.0.1:34608)
0808 14:39:22.019721 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:22.019766 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:22.019916 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:22.020866 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:22.020869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:22.020887 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:22.020973 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.020978 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.022366 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.022369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.023778 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.023785 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.025111 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.025115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.025213 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.025240 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 14:39:22.025307 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 14:39:22.025398 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1167,"mutex_wait_us":81,"spinlock_wait_cycles":8064,"thread_start_us":103,"threads_started":1}
I20260808 14:39:22.026446 28365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.019400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44500 (local address 127.19.144.67:40499)
0808 14:39:22.020254 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:22.020266 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:22.020287 (+ 21us) server_negotiation.cc:407] Connection header received
0808 14:39:22.020359 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:22.020364 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:22.020547 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:22.020729 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:22.020946 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.022205 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.023911 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.024848 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.024984 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.026102 (+ 1118us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:22.026174 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 14:39:22.026290 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":132,"threads_started":1}
I20260808 14:39:22.028558 27979 tablet_service.cc:1467] Processing CreateTablet for tablet e11112618e4b4e4e9e5bc5a874b9e77e (DEFAULT_TABLE table=test-table [id=b47276c2d1d2470987a285a0988bc896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:22.028857 28126 tablet_service.cc:1467] Processing CreateTablet for tablet e11112618e4b4e4e9e5bc5a874b9e77e (DEFAULT_TABLE table=test-table [id=b47276c2d1d2470987a285a0988bc896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:22.029206 28269 tablet_service.cc:1467] Processing CreateTablet for tablet e11112618e4b4e4e9e5bc5a874b9e77e (DEFAULT_TABLE table=test-table [id=b47276c2d1d2470987a285a0988bc896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:22.029834 27979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e11112618e4b4e4e9e5bc5a874b9e77e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:22.029969 28126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e11112618e4b4e4e9e5bc5a874b9e77e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:22.030305 28269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e11112618e4b4e4e9e5bc5a874b9e77e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:22.037002 28366 tablet_bootstrap.cc:492] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73: Bootstrap starting.
I20260808 14:39:22.037700 28367 tablet_bootstrap.cc:492] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588: Bootstrap starting.
I20260808 14:39:22.038889 28368 tablet_bootstrap.cc:492] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb: Bootstrap starting.
I20260808 14:39:22.039597 28366 tablet_bootstrap.cc:654] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:22.040207 28367 tablet_bootstrap.cc:654] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:22.040581 28366 log.cc:824] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:22.040782 28368 tablet_bootstrap.cc:654] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:22.041466 28367 log.cc:824] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:22.041601 28368 log.cc:824] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:22.042973 28366 tablet_bootstrap.cc:492] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73: No bootstrap required, opened a new log
I20260808 14:39:22.043193 28366 ts_tablet_manager.cc:1397] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:39:22.043473 28368 tablet_bootstrap.cc:492] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb: No bootstrap required, opened a new log
I20260808 14:39:22.043519 28367 tablet_bootstrap.cc:492] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588: No bootstrap required, opened a new log
I20260808 14:39:22.043684 28368 ts_tablet_manager.cc:1397] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 14:39:22.043711 28367 ts_tablet_manager.cc:1397] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 14:39:22.046710 28368 raft_consensus.cc:348] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.047006 28368 raft_consensus.cc:374] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:22.047092 28368 raft_consensus.cc:729] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43edeb6b22f84c4bb3629e52b7a6b4bb, State: Initialized, Role: FOLLOWER
I20260808 14:39:22.047652 28368 consensus_queue.cc:260] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.047814 28366 raft_consensus.cc:348] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.048130 28366 raft_consensus.cc:374] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:22.048234 28366 raft_consensus.cc:729] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dab0325fbae4f069c67092426e7cc73, State: Initialized, Role: FOLLOWER
I20260808 14:39:22.048415 28367 raft_consensus.cc:348] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.048722 28367 raft_consensus.cc:374] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:22.048751 28337 heartbeater.cc:503] Master 127.19.144.126:43957 was elected leader, sending a full tablet report...
I20260808 14:39:22.048827 28367 raft_consensus.cc:729] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3572036761bb4f29b786328b3dcb7588, State: Initialized, Role: FOLLOWER
I20260808 14:39:22.048848 28366 consensus_queue.cc:260] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.048964 28368 ts_tablet_manager.cc:1428] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 14:39:22.049314 28367 consensus_queue.cc:260] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.049721 28193 heartbeater.cc:503] Master 127.19.144.126:43957 was elected leader, sending a full tablet report...
I20260808 14:39:22.050034 28366 ts_tablet_manager.cc:1428] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260808 14:39:22.050468 28367 ts_tablet_manager.cc:1428] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260808 14:39:22.050849 28048 heartbeater.cc:503] Master 127.19.144.126:43957 was elected leader, sending a full tablet report...
I20260808 14:39:22.057003 28373 raft_consensus.cc:482] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:39:22.057171 28373 raft_consensus.cc:504] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.058269 28373 leader_election.cc:288] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3572036761bb4f29b786328b3dcb7588 (127.19.144.65:37131), 43edeb6b22f84c4bb3629e52b7a6b4bb (127.19.144.67:40499)
I20260808 14:39:22.063759 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.059189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:60283 (local address 127.19.144.65:37131)
0808 14:39:22.059335 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:22.059339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:22.059352 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:39:22.059398 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:22.059401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:22.059456 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:22.059529 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:22.059821 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.060620 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.062491 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.063124 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.063253 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.063309 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:22.063529 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 14:39:22.063626 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:39:22.063769 28083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.058529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:37131 (local address 127.19.144.66:60283)
0808 14:39:22.058648 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:22.058663 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:22.058740 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:22.059548 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:22.059551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:22.059587 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:22.059702 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.059710 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.060743 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.060746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.062353 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.062366 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.063250 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.063253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.063384 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.063412 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 14:39:22.063493 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 14:39:22.063596 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":160,"spinlock_wait_cycles":2816}
I20260808 14:39:22.063799 28365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.058918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:44391 (local address 127.19.144.67:40499)
0808 14:39:22.059044 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:22.059049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:22.059205 (+ 156us) server_negotiation.cc:407] Connection header received
0808 14:39:22.059327 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:22.059330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:22.059385 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:22.059467 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:22.059866 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.060620 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.062673 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.063288 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.063409 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.063445 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:22.063620 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 14:39:22.063692 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:39:22.063781 28375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.058776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:40499 (local address 127.19.144.66:44391)
0808 14:39:22.059111 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:22.059127 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:22.059238 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:22.059484 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:22.059486 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:22.059522 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:22.059709 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.059715 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.060743 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.060746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.062563 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.062570 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.063406 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.063409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.063512 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.063530 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 14:39:22.063594 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 14:39:22.063645 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":85,"spinlock_wait_cycles":22272,"thread_start_us":166,"threads_started":1}
I20260808 14:39:22.064580 28291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e11112618e4b4e4e9e5bc5a874b9e77e" candidate_uuid: "3dab0325fbae4f069c67092426e7cc73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" is_pre_election: true
I20260808 14:39:22.064589 27999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e11112618e4b4e4e9e5bc5a874b9e77e" candidate_uuid: "3dab0325fbae4f069c67092426e7cc73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3572036761bb4f29b786328b3dcb7588" is_pre_election: true
I20260808 14:39:22.064926 27999 raft_consensus.cc:2454] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3dab0325fbae4f069c67092426e7cc73 in term 0.
I20260808 14:39:22.064924 28291 raft_consensus.cc:2454] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3dab0325fbae4f069c67092426e7cc73 in term 0.
I20260808 14:39:22.065457 28077 leader_election.cc:302] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3572036761bb4f29b786328b3dcb7588, 3dab0325fbae4f069c67092426e7cc73; no voters:
I20260808 14:39:22.065776 28373 raft_consensus.cc:2781] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:39:22.065892 28373 raft_consensus.cc:482] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:39:22.065958 28373 raft_consensus.cc:3037] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:22.067447 28373 raft_consensus.cc:504] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.067848 28373 leader_election.cc:288] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [CANDIDATE]: Term 1 election: Requested vote from peers 3572036761bb4f29b786328b3dcb7588 (127.19.144.65:37131), 43edeb6b22f84c4bb3629e52b7a6b4bb (127.19.144.67:40499)
I20260808 14:39:22.068317 28291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e11112618e4b4e4e9e5bc5a874b9e77e" candidate_uuid: "3dab0325fbae4f069c67092426e7cc73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb"
I20260808 14:39:22.068320 27999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e11112618e4b4e4e9e5bc5a874b9e77e" candidate_uuid: "3dab0325fbae4f069c67092426e7cc73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3572036761bb4f29b786328b3dcb7588"
I20260808 14:39:22.068477 28291 raft_consensus.cc:3037] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:22.068495 27999 raft_consensus.cc:3037] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:22.069583 28291 raft_consensus.cc:2454] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3dab0325fbae4f069c67092426e7cc73 in term 1.
I20260808 14:39:22.069682 27999 raft_consensus.cc:2454] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3dab0325fbae4f069c67092426e7cc73 in term 1.
I20260808 14:39:22.070125 28077 leader_election.cc:302] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3572036761bb4f29b786328b3dcb7588, 3dab0325fbae4f069c67092426e7cc73; no voters:
I20260808 14:39:22.070374 28373 raft_consensus.cc:2781] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:22.070631 28373 raft_consensus.cc:686] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [term 1 LEADER]: Becoming Leader. State: Replica: 3dab0325fbae4f069c67092426e7cc73, State: Running, Role: LEADER
I20260808 14:39:22.070994 28373 consensus_queue.cc:237] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.074726 27834 catalog_manager.cc:5495] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3dab0325fbae4f069c67092426e7cc73 (127.19.144.66). New cstate: current_term: 1 leader_uuid: "3dab0325fbae4f069c67092426e7cc73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } health_report { overall_health: UNKNOWN } } }
I20260808 14:39:22.088796 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.084932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34616 (local address 127.19.144.66:38523)
0808 14:39:22.085078 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:22.085085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:22.085103 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:39:22.085314 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:22.085317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:22.085373 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:22.085504 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:39:22.085781 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.086297 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.087216 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.087466 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.087576 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.087869 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:39:22.088355 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:39:22.088495 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:22.088558 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 14:39:22.088618 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 14:39:22.095374 28291 raft_consensus.cc:1261] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 FOLLOWER]: Refusing update from remote peer 3dab0325fbae4f069c67092426e7cc73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:22.095384 27999 raft_consensus.cc:1261] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Refusing update from remote peer 3dab0325fbae4f069c67092426e7cc73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:22.096169 28376 consensus_queue.cc:1035] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:22.096395 28373 consensus_queue.cc:1035] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:22.103916 28379 mvcc.cc:205] Tried to move back new op lower bound from 7316275044734476288 to 7316275044645466112. Current Snapshot: MvccSnapshot[applied={T|T < 7316275044734476288}]
I20260808 14:39:22.105773 28381 mvcc.cc:205] Tried to move back new op lower bound from 7316275044734476288 to 7316275044645466112. Current Snapshot: MvccSnapshot[applied={T|T < 7316275044734476288}]
I20260808 14:39:22.112712 27836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:22.112975 27836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:22.115520 27836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d: Sending DeleteTablet for 3 replicas of tablet e11112618e4b4e4e9e5bc5a874b9e77e
I20260808 14:39:22.116501 28269 tablet_service.cc:1514] Processing DeleteTablet for tablet e11112618e4b4e4e9e5bc5a874b9e77e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:22 UTC) from {username='oryx', principal='oryx/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:44500
I20260808 14:39:22.116541 28126 tablet_service.cc:1514] Processing DeleteTablet for tablet e11112618e4b4e4e9e5bc5a874b9e77e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:22 UTC) from {username='oryx', principal='oryx/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:34608
I20260808 14:39:22.117123 27836 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40194:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 14:39:22.117333 27979 tablet_service.cc:1514] Processing DeleteTablet for tablet e11112618e4b4e4e9e5bc5a874b9e77e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:22 UTC) from {username='oryx', principal='oryx/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:33150
W20260808 14:39:22.117601 27836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:39:22.117854 28391 tablet_replica.cc:330] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb: stopping tablet replica
I20260808 14:39:22.118232 28391 raft_consensus.cc:2229] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:22.118625 28392 tablet_replica.cc:330] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588: stopping tablet replica
I20260808 14:39:22.118629 28391 raft_consensus.cc:2258] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:22.118952 28392 raft_consensus.cc:2229] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:22.119261 28392 raft_consensus.cc:2258] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:22.120437 28391 ts_tablet_manager.cc:1905] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:22.123062 27979 tablet_service.cc:1467] Processing CreateTablet for tablet f9be2859f82a4090b20e45befe6395c6 (DEFAULT_TABLE table=test-table [id=87d95c9f35254dbca70fc9fb0869fd48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:22.123363 27979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9be2859f82a4090b20e45befe6395c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:22.123333 28269 tablet_service.cc:1467] Processing CreateTablet for tablet f9be2859f82a4090b20e45befe6395c6 (DEFAULT_TABLE table=test-table [id=87d95c9f35254dbca70fc9fb0869fd48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:22.123565 28391 ts_tablet_manager.cc:1918] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:22.123616 28269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9be2859f82a4090b20e45befe6395c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:22.123660 28391 log.cc:1196] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/e11112618e4b4e4e9e5bc5a874b9e77e
I20260808 14:39:22.124023 28391 ts_tablet_manager.cc:1939] T e11112618e4b4e4e9e5bc5a874b9e77e P 43edeb6b22f84c4bb3629e52b7a6b4bb: Deleting consensus metadata
I20260808 14:39:22.124850 28390 tablet_replica.cc:330] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73: stopping tablet replica
I20260808 14:39:22.125197 28390 raft_consensus.cc:2229] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:39:22.125677 28390 raft_consensus.cc:2258] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:22.126047 28367 tablet_bootstrap.cc:492] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588: Bootstrap starting.
I20260808 14:39:22.127135 28367 tablet_bootstrap.cc:654] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:22.127511 28390 ts_tablet_manager.cc:1905] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:22.127800 28368 tablet_bootstrap.cc:492] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Bootstrap starting.
I20260808 14:39:22.128530 28367 tablet_bootstrap.cc:492] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588: No bootstrap required, opened a new log
I20260808 14:39:22.128623 28367 ts_tablet_manager.cc:1397] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260808 14:39:22.128789 28368 tablet_bootstrap.cc:654] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:22.129065 28367 raft_consensus.cc:348] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.129174 28367 raft_consensus.cc:374] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:22.129205 28367 raft_consensus.cc:729] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3572036761bb4f29b786328b3dcb7588, State: Initialized, Role: FOLLOWER
I20260808 14:39:22.129323 28367 consensus_queue.cc:260] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.129573 28367 ts_tablet_manager.cc:1428] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:39:22.130687 28126 tablet_service.cc:1467] Processing CreateTablet for tablet f9be2859f82a4090b20e45befe6395c6 (DEFAULT_TABLE table=test-table [id=87d95c9f35254dbca70fc9fb0869fd48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:22.130942 28126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9be2859f82a4090b20e45befe6395c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:22.132012 28392 ts_tablet_manager.cc:1905] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:22.133350 28366 tablet_bootstrap.cc:492] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73: Bootstrap starting.
I20260808 14:39:22.134632 28366 tablet_bootstrap.cc:654] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:22.135327 28390 ts_tablet_manager.cc:1918] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:22.135339 27823 catalog_manager.cc:4841] TS 43edeb6b22f84c4bb3629e52b7a6b4bb (127.19.144.67:40499): tablet e11112618e4b4e4e9e5bc5a874b9e77e (table test-table [id=b47276c2d1d2470987a285a0988bc896]) successfully deleted
I20260808 14:39:22.135543 28390 log.cc:1196] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/e11112618e4b4e4e9e5bc5a874b9e77e
I20260808 14:39:22.135953 28390 ts_tablet_manager.cc:1939] T e11112618e4b4e4e9e5bc5a874b9e77e P 3dab0325fbae4f069c67092426e7cc73: Deleting consensus metadata
I20260808 14:39:22.136094 28366 tablet_bootstrap.cc:492] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73: No bootstrap required, opened a new log
I20260808 14:39:22.136197 28366 ts_tablet_manager.cc:1397] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:39:22.136723 28366 raft_consensus.cc:348] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.136876 28366 raft_consensus.cc:374] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:22.136934 28366 raft_consensus.cc:729] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dab0325fbae4f069c67092426e7cc73, State: Initialized, Role: FOLLOWER
I20260808 14:39:22.137071 28366 consensus_queue.cc:260] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.137421 28366 ts_tablet_manager.cc:1428] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 14:39:22.138339 28368 tablet_bootstrap.cc:492] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb: No bootstrap required, opened a new log
I20260808 14:39:22.138443 28368 ts_tablet_manager.cc:1397] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260808 14:39:22.139037 28368 raft_consensus.cc:348] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.139181 28368 raft_consensus.cc:374] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:22.139231 28368 raft_consensus.cc:729] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43edeb6b22f84c4bb3629e52b7a6b4bb, State: Initialized, Role: FOLLOWER
I20260808 14:39:22.139379 28368 consensus_queue.cc:260] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.139696 28368 ts_tablet_manager.cc:1428] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:39:22.139853 27820 catalog_manager.cc:4841] TS 3dab0325fbae4f069c67092426e7cc73 (127.19.144.66:38523): tablet e11112618e4b4e4e9e5bc5a874b9e77e (table test-table [id=b47276c2d1d2470987a285a0988bc896]) successfully deleted
I20260808 14:39:22.140841 28392 ts_tablet_manager.cc:1918] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:22.140988 28392 log.cc:1196] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/wal/wals/e11112618e4b4e4e9e5bc5a874b9e77e
I20260808 14:39:22.141388 28392 ts_tablet_manager.cc:1939] T e11112618e4b4e4e9e5bc5a874b9e77e P 3572036761bb4f29b786328b3dcb7588: Deleting consensus metadata
I20260808 14:39:22.142802 27823 catalog_manager.cc:4841] TS 3572036761bb4f29b786328b3dcb7588 (127.19.144.65:37131): tablet e11112618e4b4e4e9e5bc5a874b9e77e (table test-table [id=b47276c2d1d2470987a285a0988bc896]) successfully deleted
I20260808 14:39:22.165584 28372 raft_consensus.cc:482] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:39:22.165793 28372 raft_consensus.cc:504] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.167061 28372 leader_election.cc:288] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3572036761bb4f29b786328b3dcb7588 (127.19.144.65:37131), 3dab0325fbae4f069c67092426e7cc73 (127.19.144.66:38523)
I20260808 14:39:22.172093 28227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.167283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:37131 (local address 127.19.144.67:53197)
0808 14:39:22.167576 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:22.167595 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:22.167697 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:22.168014 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:22.168018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:22.168038 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:22.168200 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.168209 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.169272 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.169276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.170719 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.170732 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.171611 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.171615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.171736 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.171762 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 14:39:22.171846 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 14:39:22.171926 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":124}
I20260808 14:39:22.172109 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.167427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:53197 (local address 127.19.144.65:37131)
0808 14:39:22.167679 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:22.167684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:22.167701 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:39:22.167836 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:22.167839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:22.167893 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:22.167998 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:22.168336 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.169141 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.170872 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.171488 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.171614 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.171669 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:22.171924 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 14:39:22.172000 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260808 14:39:22.172581 27999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9be2859f82a4090b20e45befe6395c6" candidate_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3572036761bb4f29b786328b3dcb7588" is_pre_election: true
I20260808 14:39:22.172793 28396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.167330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:38523 (local address 127.19.144.67:57417)
0808 14:39:22.167950 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:22.167964 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:22.168111 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:22.168434 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:22.168437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:22.168453 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:22.168582 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.168588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.169322 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.169325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.170719 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.170732 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.172360 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.172364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.172482 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.172506 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:39:22.172575 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 14:39:22.172641 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":28,"thread_start_us":113,"threads_started":1}
I20260808 14:39:22.172807 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.167604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:57417 (local address 127.19.144.66:38523)
0808 14:39:22.167746 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:22.167751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:22.168039 (+ 288us) server_negotiation.cc:407] Connection header received
0808 14:39:22.168238 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:22.168244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:22.168306 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:22.168418 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:22.168688 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.169179 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.170869 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.171491 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.171614 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.171669 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:22.172603 (+ 934us) server_negotiation.cc:299] Negotiation successful
0808 14:39:22.172689 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:39:22.172799 27999 raft_consensus.cc:2454] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43edeb6b22f84c4bb3629e52b7a6b4bb in term 0.
I20260808 14:39:22.173230 28222 leader_election.cc:302] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3572036761bb4f29b786328b3dcb7588, 43edeb6b22f84c4bb3629e52b7a6b4bb; no voters:
I20260808 14:39:22.173550 28146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9be2859f82a4090b20e45befe6395c6" candidate_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dab0325fbae4f069c67092426e7cc73" is_pre_election: true
I20260808 14:39:22.173770 28372 raft_consensus.cc:2781] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:39:22.173830 28372 raft_consensus.cc:482] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:39:22.173871 28372 raft_consensus.cc:3037] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:22.173852 28146 raft_consensus.cc:2454] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43edeb6b22f84c4bb3629e52b7a6b4bb in term 0.
I20260808 14:39:22.175102 28372 raft_consensus.cc:504] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.175462 28372 leader_election.cc:288] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [CANDIDATE]: Term 1 election: Requested vote from peers 3572036761bb4f29b786328b3dcb7588 (127.19.144.65:37131), 3dab0325fbae4f069c67092426e7cc73 (127.19.144.66:38523)
I20260808 14:39:22.175871 27999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9be2859f82a4090b20e45befe6395c6" candidate_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3572036761bb4f29b786328b3dcb7588"
I20260808 14:39:22.175985 27999 raft_consensus.cc:3037] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:22.176968 27999 raft_consensus.cc:2454] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43edeb6b22f84c4bb3629e52b7a6b4bb in term 1.
I20260808 14:39:22.177280 28146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9be2859f82a4090b20e45befe6395c6" candidate_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dab0325fbae4f069c67092426e7cc73"
I20260808 14:39:22.177309 28222 leader_election.cc:302] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3572036761bb4f29b786328b3dcb7588, 43edeb6b22f84c4bb3629e52b7a6b4bb; no voters:
I20260808 14:39:22.177412 28146 raft_consensus.cc:3037] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:22.178447 28146 raft_consensus.cc:2454] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43edeb6b22f84c4bb3629e52b7a6b4bb in term 1.
I20260808 14:39:22.178696 28372 raft_consensus.cc:2781] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:22.178978 28372 raft_consensus.cc:686] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 LEADER]: Becoming Leader. State: Replica: 43edeb6b22f84c4bb3629e52b7a6b4bb, State: Running, Role: LEADER
I20260808 14:39:22.179271 28372 consensus_queue.cc:237] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.183151 27836 catalog_manager.cc:5495] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 43edeb6b22f84c4bb3629e52b7a6b4bb (127.19.144.67). New cstate: current_term: 1 leader_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } health_report { overall_health: HEALTHY } } }
W20260808 14:39:22.196380 28049 tablet.cc:2369] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:39:22.196805 28338 tablet.cc:2369] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:39:22.205456 28194 tablet.cc:2369] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 14:39:22 dist-test-slave-p3t5 krb5kdc[27790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199961, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.126@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM
I20260808 14:39:22.229421 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.219604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40204 (local address 127.19.144.126:43957)
0808 14:39:22.219780 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:22.219786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:22.219981 (+ 195us) server_negotiation.cc:407] Connection header received
0808 14:39:22.220153 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:22.220157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:22.220206 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:22.220297 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:22.221031 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.221573 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.222462 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.222680 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.222791 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.225754 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:22.225757 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:22.225759 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:22.225783 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:22.227431 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:22.228025 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:22.228027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:22.228029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:22.228105 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:22.228428 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:22.228430 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:22.228431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:22.228611 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:22.228687 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:22.229067 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 14:39:22.229266 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":59}
I20260808 14:39:22.229442 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.219533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43957 (local address 127.0.0.1:40204)
0808 14:39:22.219881 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:22.219894 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:22.220049 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:22.220323 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:22.220326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:22.220782 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:22.220888 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.220896 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.221740 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.221744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.222325 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.222334 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.222799 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.222802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.222879 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.223539 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:22.223551 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:22.225534 (+ 1983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:22.227557 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:22.227570 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:22.227578 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:22.227909 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:22.228203 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:22.228206 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:22.228208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:22.228329 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:22.228725 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:22.228732 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:22.228877 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 14:39:22.229007 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 14:39:22.229190 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":140,"threads_started":1}
I20260808 14:39:22.233789 27836 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:40204:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 14:39:22.234601 27836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:39:22.240227 28269 tablet_service.cc:1467] Processing CreateTablet for tablet d330d3381c65459d831222d5dd7bda17 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bc814acccfe4dd098363336d66086cd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:39:22.240227 28126 tablet_service.cc:1467] Processing CreateTablet for tablet d330d3381c65459d831222d5dd7bda17 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bc814acccfe4dd098363336d66086cd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:39:22.240257 27979 tablet_service.cc:1467] Processing CreateTablet for tablet d330d3381c65459d831222d5dd7bda17 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bc814acccfe4dd098363336d66086cd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 14:39:22.240707 28126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d330d3381c65459d831222d5dd7bda17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:22.240707 28269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d330d3381c65459d831222d5dd7bda17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:22.240732 27979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d330d3381c65459d831222d5dd7bda17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:22.243953 28366 tablet_bootstrap.cc:492] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73: Bootstrap starting.
I20260808 14:39:22.244465 28367 tablet_bootstrap.cc:492] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588: Bootstrap starting.
I20260808 14:39:22.245043 28366 tablet_bootstrap.cc:654] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:22.245457 28367 tablet_bootstrap.cc:654] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:22.245965 28368 tablet_bootstrap.cc:492] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Bootstrap starting.
I20260808 14:39:22.246999 28368 tablet_bootstrap.cc:654] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:22.247066 28367 tablet_bootstrap.cc:492] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588: No bootstrap required, opened a new log
I20260808 14:39:22.247110 28366 tablet_bootstrap.cc:492] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73: No bootstrap required, opened a new log
I20260808 14:39:22.247149 28367 ts_tablet_manager.cc:1397] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260808 14:39:22.247197 28366 ts_tablet_manager.cc:1397] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:39:22.247677 28367 raft_consensus.cc:348] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.247673 28366 raft_consensus.cc:348] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.247808 28367 raft_consensus.cc:374] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:22.247824 28366 raft_consensus.cc:374] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:22.247864 28366 raft_consensus.cc:729] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dab0325fbae4f069c67092426e7cc73, State: Initialized, Role: FOLLOWER
I20260808 14:39:22.247852 28367 raft_consensus.cc:729] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3572036761bb4f29b786328b3dcb7588, State: Initialized, Role: FOLLOWER
I20260808 14:39:22.248003 28366 consensus_queue.cc:260] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.248009 28367 consensus_queue.cc:260] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.248318 28366 ts_tablet_manager.cc:1428] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:39:22.248324 28367 ts_tablet_manager.cc:1428] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 14:39:22.248605 28385 tablet_replica.cc:439] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } } }
I20260808 14:39:22.248636 28374 tablet_replica.cc:439] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } } }
I20260808 14:39:22.248777 28374 tablet_replica.cc:442] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:39:22.248778 28385 tablet_replica.cc:442] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:39:22.250365 28368 tablet_bootstrap.cc:492] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb: No bootstrap required, opened a new log
I20260808 14:39:22.250466 28368 ts_tablet_manager.cc:1397] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 14:39:22.250970 28368 raft_consensus.cc:348] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.251097 28368 raft_consensus.cc:374] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:22.251134 28368 raft_consensus.cc:729] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43edeb6b22f84c4bb3629e52b7a6b4bb, State: Initialized, Role: FOLLOWER
I20260808 14:39:22.251271 28368 consensus_queue.cc:260] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.251596 28368 ts_tablet_manager.cc:1428] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 14:39:22.251595 28372 tablet_replica.cc:439] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } } }
I20260808 14:39:22.251785 28372 tablet_replica.cc:442] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:39:22.354952 28372 raft_consensus.cc:482] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:39:22.355129 28372 raft_consensus.cc:504] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.355746 28372 leader_election.cc:288] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3572036761bb4f29b786328b3dcb7588 (127.19.144.65:37131), 3dab0325fbae4f069c67092426e7cc73 (127.19.144.66:38523)
I20260808 14:39:22.356309 28146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d330d3381c65459d831222d5dd7bda17" candidate_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dab0325fbae4f069c67092426e7cc73" is_pre_election: true
I20260808 14:39:22.356331 27999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d330d3381c65459d831222d5dd7bda17" candidate_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3572036761bb4f29b786328b3dcb7588" is_pre_election: true
I20260808 14:39:22.356500 28146 raft_consensus.cc:2454] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43edeb6b22f84c4bb3629e52b7a6b4bb in term 0.
I20260808 14:39:22.356499 27999 raft_consensus.cc:2454] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43edeb6b22f84c4bb3629e52b7a6b4bb in term 0.
I20260808 14:39:22.356918 28219 leader_election.cc:302] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dab0325fbae4f069c67092426e7cc73, 43edeb6b22f84c4bb3629e52b7a6b4bb; no voters:
I20260808 14:39:22.357169 28372 raft_consensus.cc:2781] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:39:22.357256 28372 raft_consensus.cc:482] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:39:22.357292 28372 raft_consensus.cc:3037] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:22.358412 28372 raft_consensus.cc:504] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.358757 28372 leader_election.cc:288] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [CANDIDATE]: Term 1 election: Requested vote from peers 3572036761bb4f29b786328b3dcb7588 (127.19.144.65:37131), 3dab0325fbae4f069c67092426e7cc73 (127.19.144.66:38523)
I20260808 14:39:22.359194 27999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d330d3381c65459d831222d5dd7bda17" candidate_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3572036761bb4f29b786328b3dcb7588"
I20260808 14:39:22.359195 28146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d330d3381c65459d831222d5dd7bda17" candidate_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dab0325fbae4f069c67092426e7cc73"
I20260808 14:39:22.359333 27999 raft_consensus.cc:3037] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:22.359333 28146 raft_consensus.cc:3037] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:22.360528 27999 raft_consensus.cc:2454] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43edeb6b22f84c4bb3629e52b7a6b4bb in term 1.
I20260808 14:39:22.360528 28146 raft_consensus.cc:2454] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43edeb6b22f84c4bb3629e52b7a6b4bb in term 1.
I20260808 14:39:22.360875 28222 leader_election.cc:302] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3572036761bb4f29b786328b3dcb7588, 43edeb6b22f84c4bb3629e52b7a6b4bb; no voters:
I20260808 14:39:22.361117 28372 raft_consensus.cc:2781] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:22.361225 28372 raft_consensus.cc:686] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 LEADER]: Becoming Leader. State: Replica: 43edeb6b22f84c4bb3629e52b7a6b4bb, State: Running, Role: LEADER
I20260808 14:39:22.361362 28372 consensus_queue.cc:237] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.362217 28397 tablet_replica.cc:439] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb: TxnStatusTablet state changed. Reason: New leader 43edeb6b22f84c4bb3629e52b7a6b4bb. Latest consensus state: current_term: 1 leader_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } } }
I20260808 14:39:22.362339 28397 tablet_replica.cc:442] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:39:22.362655 28409 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 14:39:22.362888 28409 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 14:39:22.363438 27834 catalog_manager.cc:5495] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 43edeb6b22f84c4bb3629e52b7a6b4bb (127.19.144.67). New cstate: current_term: 1 leader_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } health_report { overall_health: HEALTHY } } }
I20260808 14:39:22.381075 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.376341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:40499 (local address 127.0.0.1:44506)
0808 14:39:22.376691 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:22.376710 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:22.376825 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:22.377196 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:22.377199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:22.377206 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:39:22.377305 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.377309 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.378464 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.378466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.379105 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.379112 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.379831 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.379834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.379909 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.379961 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:39:22.380816 (+ 855us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:39:22.380827 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 14:39:22.380871 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 14:39:22.380951 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240}
I20260808 14:39:22.381114 28365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.376496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44506 (local address 127.19.144.67:40499)
0808 14:39:22.376651 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:22.376656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:22.376790 (+ 134us) server_negotiation.cc:407] Connection header received
0808 14:39:22.376969 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:22.376973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:22.377052 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:22.377172 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:39:22.377415 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.378344 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.379222 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.379667 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.379826 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.380108 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:39:22.380707 (+ 599us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:39:22.380849 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:22.380897 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 14:39:22.380961 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:39:22.384414 27999 raft_consensus.cc:1261] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Refusing update from remote peer 43edeb6b22f84c4bb3629e52b7a6b4bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:22.384665 28146 raft_consensus.cc:1261] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73 [term 1 FOLLOWER]: Refusing update from remote peer 43edeb6b22f84c4bb3629e52b7a6b4bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:22.385265 28372 consensus_queue.cc:1035] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:22.385525 28397 consensus_queue.cc:1035] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:22.387688 28374 tablet_replica.cc:439] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588: TxnStatusTablet state changed. Reason: New leader 43edeb6b22f84c4bb3629e52b7a6b4bb. Latest consensus state: current_term: 1 leader_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } } }
I20260808 14:39:22.387820 28374 tablet_replica.cc:442] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:39:22.388298 28385 tablet_replica.cc:439] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73: TxnStatusTablet state changed. Reason: New leader 43edeb6b22f84c4bb3629e52b7a6b4bb. Latest consensus state: current_term: 1 leader_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } } }
I20260808 14:39:22.388443 28385 tablet_replica.cc:442] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:39:22.390386 28397 tablet_replica.cc:439] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } } }
I20260808 14:39:22.390527 28397 tablet_replica.cc:442] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:39:22.390635 28380 mvcc.cc:205] Tried to move back new op lower bound from 7316275045920641024 to 7316275045833838592. Current Snapshot: MvccSnapshot[applied={T|T < 7316275045920641024}]
I20260808 14:39:22.391309 28385 tablet_replica.cc:439] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } } }
I20260808 14:39:22.391448 28385 tablet_replica.cc:442] T d330d3381c65459d831222d5dd7bda17 P 3dab0325fbae4f069c67092426e7cc73: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:39:22.393018 28372 tablet_replica.cc:439] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } } }
I20260808 14:39:22.393152 28372 tablet_replica.cc:442] T d330d3381c65459d831222d5dd7bda17 P 43edeb6b22f84c4bb3629e52b7a6b4bb: This TxnStatusTablet replica's current role is: LEADER
I20260808 14:39:22.395534 28374 tablet_replica.cc:439] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } } }
I20260808 14:39:22.395663 28374 tablet_replica.cc:442] T d330d3381c65459d831222d5dd7bda17 P 3572036761bb4f29b786328b3dcb7588: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 14:39:22.400977 28365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.397649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44512 (local address 127.19.144.67:40499)
0808 14:39:22.397761 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:22.397766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:22.397845 (+ 79us) server_negotiation.cc:407] Connection header received
0808 14:39:22.397998 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:22.398001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:22.398056 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:22.398141 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:39:22.398421 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.399006 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.399867 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.400102 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.400215 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.400498 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:39:22.400576 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:39:22.400658 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:22.400755 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 14:39:22.400811 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 14:39:22.414196 28365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.409578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:54897 (local address 127.19.144.67:40499)
0808 14:39:22.409786 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:22.409789 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:22.409864 (+ 75us) server_negotiation.cc:407] Connection header received
0808 14:39:22.410027 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:22.410029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:22.410072 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:22.410129 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:39:22.411110 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.411599 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.412982 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.413197 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.413287 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.413723 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:39:22.413799 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:39:22.413868 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:22.414002 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 14:39:22.414050 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260808 14:39:22.414247 28343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.409549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:40499 (local address 127.19.144.67:54897)
0808 14:39:22.409770 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:22.409789 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:22.409912 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:22.410189 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:22.410193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:22.410201 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 14:39:22.410960 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.410972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.411712 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.411716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.412827 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:22.412839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.413334 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.413340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:22.413454 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.413519 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 14:39:22.413896 (+ 377us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 14:39:22.413904 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 14:39:22.413982 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 14:39:22.414078 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 14:39:22.423556 27999 raft_consensus.cc:1261] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Refusing update from remote peer 43edeb6b22f84c4bb3629e52b7a6b4bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:22.423555 28146 raft_consensus.cc:1261] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73 [term 1 FOLLOWER]: Refusing update from remote peer 43edeb6b22f84c4bb3629e52b7a6b4bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:22.423983 28372 consensus_queue.cc:1035] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:22.424160 28414 consensus_queue.cc:1035] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:22.469538 27836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:22.469779 27836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:22.472127 27836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d: Sending DeleteTablet for 3 replicas of tablet f9be2859f82a4090b20e45befe6395c6
I20260808 14:39:22.472800 27979 tablet_service.cc:1514] Processing DeleteTablet for tablet f9be2859f82a4090b20e45befe6395c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:22 UTC) from {username='oryx', principal='oryx/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:33150
I20260808 14:39:22.472898 28269 tablet_service.cc:1514] Processing DeleteTablet for tablet f9be2859f82a4090b20e45befe6395c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:22 UTC) from {username='oryx', principal='oryx/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:44500
I20260808 14:39:22.473017 28392 tablet_replica.cc:330] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588: stopping tablet replica
I20260808 14:39:22.473042 28126 tablet_service.cc:1514] Processing DeleteTablet for tablet f9be2859f82a4090b20e45befe6395c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:22 UTC) from {username='oryx', principal='oryx/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:34608
I20260808 14:39:22.473140 28392 raft_consensus.cc:2229] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:22.473275 28392 raft_consensus.cc:2258] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:22.473488 28391 tablet_replica.cc:330] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb: stopping tablet replica
I20260808 14:39:22.473651 28391 raft_consensus.cc:2229] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:39:22.473984 28391 raft_consensus.cc:2258] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:22.473989 28390 tablet_replica.cc:330] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73: stopping tablet replica
I20260808 14:39:22.474174 28390 raft_consensus.cc:2229] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:22.474332 28390 raft_consensus.cc:2258] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:22.474920 28392 ts_tablet_manager.cc:1905] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:22.475412 28390 ts_tablet_manager.cc:1905] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:22.478149 28392 ts_tablet_manager.cc:1918] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:39:22.478255 28392 log.cc:1196] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/wal/wals/f9be2859f82a4090b20e45befe6395c6
I20260808 14:39:22.478550 28392 ts_tablet_manager.cc:1939] T f9be2859f82a4090b20e45befe6395c6 P 3572036761bb4f29b786328b3dcb7588: Deleting consensus metadata
I20260808 14:39:22.479202 28390 ts_tablet_manager.cc:1918] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:39:22.479313 28390 log.cc:1196] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/f9be2859f82a4090b20e45befe6395c6
I20260808 14:39:22.479626 28390 ts_tablet_manager.cc:1939] T f9be2859f82a4090b20e45befe6395c6 P 3dab0325fbae4f069c67092426e7cc73: Deleting consensus metadata
I20260808 14:39:22.479707 27823 catalog_manager.cc:4841] TS 3572036761bb4f29b786328b3dcb7588 (127.19.144.65:37131): tablet f9be2859f82a4090b20e45befe6395c6 (table test-table [id=87d95c9f35254dbca70fc9fb0869fd48]) successfully deleted
I20260808 14:39:22.480595 28391 ts_tablet_manager.cc:1905] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:22.480573 27820 catalog_manager.cc:4841] TS 3dab0325fbae4f069c67092426e7cc73 (127.19.144.66:38523): tablet f9be2859f82a4090b20e45befe6395c6 (table test-table [id=87d95c9f35254dbca70fc9fb0869fd48]) successfully deleted
I20260808 14:39:22.483610 28391 ts_tablet_manager.cc:1918] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:39:22.483703 28391 log.cc:1196] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/f9be2859f82a4090b20e45befe6395c6
I20260808 14:39:22.483985 28391 ts_tablet_manager.cc:1939] T f9be2859f82a4090b20e45befe6395c6 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Deleting consensus metadata
I20260808 14:39:22.484769 27823 catalog_manager.cc:4841] TS 43edeb6b22f84c4bb3629e52b7a6b4bb (127.19.144.67:40499): tablet f9be2859f82a4090b20e45befe6395c6 (table test-table [id=87d95c9f35254dbca70fc9fb0869fd48]) successfully deleted
Aug 08 14:39:22 dist-test-slave-p3t5 krb5kdc[27790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 14:39:22 dist-test-slave-p3t5 krb5kdc[27790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:22 dist-test-slave-p3t5 krb5kdc[27790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 14:39:22.509858 27887 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:22.502515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40218 (local address 127.19.144.126:43957)
0808 14:39:22.502673 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:22.502681 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:22.502899 (+ 218us) server_negotiation.cc:407] Connection header received
0808 14:39:22.503093 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:22.503097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:22.503157 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:22.503259 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:22.504118 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.504709 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.505809 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.506076 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.506205 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.509735 (+ 3530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40218: BlockingRecv error: recv got EOF from 127.0.0.1:40218 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 14:39:22 dist-test-slave-p3t5 krb5kdc[27790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM
I20260808 14:39:22.521924 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.512020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40220 (local address 127.19.144.126:43957)
0808 14:39:22.512173 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:22.512180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:22.512353 (+ 173us) server_negotiation.cc:407] Connection header received
0808 14:39:22.512473 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:22.512476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:22.512519 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:22.512592 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:22.513343 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.513979 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.515053 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.515287 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.515373 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.518551 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:22.518555 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:22.518557 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:22.518585 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:22.520097 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:22.520560 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:22.520564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:22.520568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:22.520654 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:22.520952 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:22.520955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:22.520957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:22.521156 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:22.521303 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:22.521565 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 14:39:22.521746 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 14:39:22.525058 27836 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:39:22.525552 27836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:39:22.531226 27979 tablet_service.cc:1467] Processing CreateTablet for tablet 663e229217124d2e9527f358bd320ea8 (DEFAULT_TABLE table=test-table [id=f68dcbe54fa04079b580ef36e8b40d0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:22.531477 28269 tablet_service.cc:1467] Processing CreateTablet for tablet 663e229217124d2e9527f358bd320ea8 (DEFAULT_TABLE table=test-table [id=f68dcbe54fa04079b580ef36e8b40d0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:22.531643 27979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 663e229217124d2e9527f358bd320ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:22.531807 28269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 663e229217124d2e9527f358bd320ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:22.533896 28368 tablet_bootstrap.cc:492] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Bootstrap starting.
I20260808 14:39:22.534545 28367 tablet_bootstrap.cc:492] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588: Bootstrap starting.
I20260808 14:39:22.534906 28368 tablet_bootstrap.cc:654] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:22.535470 28367 tablet_bootstrap.cc:654] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:22.535571 28126 tablet_service.cc:1467] Processing CreateTablet for tablet 663e229217124d2e9527f358bd320ea8 (DEFAULT_TABLE table=test-table [id=f68dcbe54fa04079b580ef36e8b40d0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:22.535912 28126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 663e229217124d2e9527f358bd320ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:22.536813 28367 tablet_bootstrap.cc:492] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588: No bootstrap required, opened a new log
I20260808 14:39:22.536933 28367 ts_tablet_manager.cc:1397] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 14:39:22.537505 28367 raft_consensus.cc:348] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.537698 28367 raft_consensus.cc:374] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:22.537741 28367 raft_consensus.cc:729] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3572036761bb4f29b786328b3dcb7588, State: Initialized, Role: FOLLOWER
I20260808 14:39:22.537889 28367 consensus_queue.cc:260] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.538271 28367 ts_tablet_manager.cc:1428] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 14:39:22.538781 28366 tablet_bootstrap.cc:492] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73: Bootstrap starting.
I20260808 14:39:22.539930 28366 tablet_bootstrap.cc:654] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:22.541054 28368 tablet_bootstrap.cc:492] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb: No bootstrap required, opened a new log
I20260808 14:39:22.541163 28368 ts_tablet_manager.cc:1397] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260808 14:39:22.541352 28366 tablet_bootstrap.cc:492] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73: No bootstrap required, opened a new log
I20260808 14:39:22.541447 28366 ts_tablet_manager.cc:1397] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:39:22.541759 28368 raft_consensus.cc:348] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.541904 28368 raft_consensus.cc:374] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:22.541941 28368 raft_consensus.cc:729] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43edeb6b22f84c4bb3629e52b7a6b4bb, State: Initialized, Role: FOLLOWER
I20260808 14:39:22.541885 28366 raft_consensus.cc:348] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.542004 28366 raft_consensus.cc:374] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:22.542038 28366 raft_consensus.cc:729] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dab0325fbae4f069c67092426e7cc73, State: Initialized, Role: FOLLOWER
I20260808 14:39:22.542081 28368 consensus_queue.cc:260] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.542151 28366 consensus_queue.cc:260] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.542363 28368 ts_tablet_manager.cc:1428] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:39:22.542407 28366 ts_tablet_manager.cc:1428] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:39:22.658267 28385 raft_consensus.cc:482] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:39:22.658454 28385 raft_consensus.cc:504] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.658951 28385 leader_election.cc:288] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3572036761bb4f29b786328b3dcb7588 (127.19.144.65:37131), 43edeb6b22f84c4bb3629e52b7a6b4bb (127.19.144.67:40499)
I20260808 14:39:22.659551 27999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "663e229217124d2e9527f358bd320ea8" candidate_uuid: "3dab0325fbae4f069c67092426e7cc73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3572036761bb4f29b786328b3dcb7588" is_pre_election: true
I20260808 14:39:22.659592 28291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "663e229217124d2e9527f358bd320ea8" candidate_uuid: "3dab0325fbae4f069c67092426e7cc73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" is_pre_election: true
I20260808 14:39:22.659739 28291 raft_consensus.cc:2454] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3dab0325fbae4f069c67092426e7cc73 in term 0.
I20260808 14:39:22.659739 27999 raft_consensus.cc:2454] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3dab0325fbae4f069c67092426e7cc73 in term 0.
I20260808 14:39:22.660086 28077 leader_election.cc:302] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3572036761bb4f29b786328b3dcb7588, 3dab0325fbae4f069c67092426e7cc73; no voters:
I20260808 14:39:22.660326 28385 raft_consensus.cc:2781] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:39:22.660408 28385 raft_consensus.cc:482] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:39:22.660466 28385 raft_consensus.cc:3037] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:22.661595 28385 raft_consensus.cc:504] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.661999 28385 leader_election.cc:288] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [CANDIDATE]: Term 1 election: Requested vote from peers 3572036761bb4f29b786328b3dcb7588 (127.19.144.65:37131), 43edeb6b22f84c4bb3629e52b7a6b4bb (127.19.144.67:40499)
I20260808 14:39:22.662473 28291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "663e229217124d2e9527f358bd320ea8" candidate_uuid: "3dab0325fbae4f069c67092426e7cc73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb"
I20260808 14:39:22.662483 27999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "663e229217124d2e9527f358bd320ea8" candidate_uuid: "3dab0325fbae4f069c67092426e7cc73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3572036761bb4f29b786328b3dcb7588"
I20260808 14:39:22.662621 27999 raft_consensus.cc:3037] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:22.662622 28291 raft_consensus.cc:3037] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:22.663597 28291 raft_consensus.cc:2454] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3dab0325fbae4f069c67092426e7cc73 in term 1.
I20260808 14:39:22.663832 27999 raft_consensus.cc:2454] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3dab0325fbae4f069c67092426e7cc73 in term 1.
I20260808 14:39:22.663932 28077 leader_election.cc:302] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dab0325fbae4f069c67092426e7cc73, 43edeb6b22f84c4bb3629e52b7a6b4bb; no voters:
I20260808 14:39:22.664135 28385 raft_consensus.cc:2781] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:22.664335 28385 raft_consensus.cc:686] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [term 1 LEADER]: Becoming Leader. State: Replica: 3dab0325fbae4f069c67092426e7cc73, State: Running, Role: LEADER
I20260808 14:39:22.664499 28385 consensus_queue.cc:237] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.666565 27835 catalog_manager.cc:5495] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3dab0325fbae4f069c67092426e7cc73 (127.19.144.66). New cstate: current_term: 1 leader_uuid: "3dab0325fbae4f069c67092426e7cc73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } health_report { overall_health: UNKNOWN } } }
I20260808 14:39:22.707448 28439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.702875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34624 (local address 127.19.144.66:38523)
0808 14:39:22.703151 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:22.703157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:22.703173 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:39:22.703243 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:22.703246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:22.703306 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:22.703415 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:39:22.703741 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.704564 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.705852 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.706199 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.706332 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.706642 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:39:22.706778 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:39:22.707183 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:22.707231 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 14:39:22.707291 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":76,"threads_started":1}
I20260808 14:39:22.709908 28291 raft_consensus.cc:1261] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 FOLLOWER]: Refusing update from remote peer 3dab0325fbae4f069c67092426e7cc73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:22.710064 27999 raft_consensus.cc:1261] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Refusing update from remote peer 3dab0325fbae4f069c67092426e7cc73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:22.710685 28385 consensus_queue.cc:1035] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:22.711014 28436 consensus_queue.cc:1035] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:22.721747 27835 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:22.721930 27835 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:22.726073 27835 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d: Sending DeleteTablet for 3 replicas of tablet 663e229217124d2e9527f358bd320ea8
I20260808 14:39:22.726691 28269 tablet_service.cc:1514] Processing DeleteTablet for tablet 663e229217124d2e9527f358bd320ea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:22 UTC) from {username='oryx', principal='oryx/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:44500
I20260808 14:39:22.726864 27979 tablet_service.cc:1514] Processing DeleteTablet for tablet 663e229217124d2e9527f358bd320ea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:22 UTC) from {username='oryx', principal='oryx/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:33150
I20260808 14:39:22.726922 28391 tablet_replica.cc:330] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb: stopping tablet replica
I20260808 14:39:22.727041 28392 tablet_replica.cc:330] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588: stopping tablet replica
I20260808 14:39:22.726998 28126 tablet_service.cc:1514] Processing DeleteTablet for tablet 663e229217124d2e9527f358bd320ea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:22 UTC) from {username='oryx', principal='oryx/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:34608
I20260808 14:39:22.727043 28391 raft_consensus.cc:2229] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:22.727135 28392 raft_consensus.cc:2229] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:22.727175 28391 raft_consensus.cc:2258] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:22.727257 28392 raft_consensus.cc:2258] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:22.727432 28390 tablet_replica.cc:330] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73: stopping tablet replica
I20260808 14:39:22.727564 28390 raft_consensus.cc:2229] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:39:22.727766 28390 raft_consensus.cc:2258] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:22.728116 28392 ts_tablet_manager.cc:1905] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:22.728591 28391 ts_tablet_manager.cc:1905] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:22.728811 28390 ts_tablet_manager.cc:1905] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:22.730921 28392 ts_tablet_manager.cc:1918] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:22.731002 28392 log.cc:1196] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-0/wal/wals/663e229217124d2e9527f358bd320ea8
I20260808 14:39:22.731274 28392 ts_tablet_manager.cc:1939] T 663e229217124d2e9527f358bd320ea8 P 3572036761bb4f29b786328b3dcb7588: Deleting consensus metadata
I20260808 14:39:22.731590 28390 ts_tablet_manager.cc:1918] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:22.731621 28391 ts_tablet_manager.cc:1918] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:22.731662 28390 log.cc:1196] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/663e229217124d2e9527f358bd320ea8
I20260808 14:39:22.731696 28391 log.cc:1196] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/663e229217124d2e9527f358bd320ea8
I20260808 14:39:22.731954 28391 ts_tablet_manager.cc:1939] T 663e229217124d2e9527f358bd320ea8 P 43edeb6b22f84c4bb3629e52b7a6b4bb: Deleting consensus metadata
I20260808 14:39:22.731966 28390 ts_tablet_manager.cc:1939] T 663e229217124d2e9527f358bd320ea8 P 3dab0325fbae4f069c67092426e7cc73: Deleting consensus metadata
I20260808 14:39:22.732647 27820 catalog_manager.cc:4841] TS 3dab0325fbae4f069c67092426e7cc73 (127.19.144.66:38523): tablet 663e229217124d2e9527f358bd320ea8 (table test-table [id=f68dcbe54fa04079b580ef36e8b40d0d]) successfully deleted
I20260808 14:39:22.733071 27835 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 14:39:22.733351 27823 catalog_manager.cc:4841] TS 3572036761bb4f29b786328b3dcb7588 (127.19.144.65:37131): tablet 663e229217124d2e9527f358bd320ea8 (table test-table [id=f68dcbe54fa04079b580ef36e8b40d0d]) successfully deleted
I20260808 14:39:22.733492 27823 catalog_manager.cc:4841] TS 43edeb6b22f84c4bb3629e52b7a6b4bb (127.19.144.67:40499): tablet 663e229217124d2e9527f358bd320ea8 (table test-table [id=f68dcbe54fa04079b580ef36e8b40d0d]) successfully deleted
W20260808 14:39:22.733541 27835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:39:22.738770 28269 tablet_service.cc:1467] Processing CreateTablet for tablet 5f23ccf7f8b541c59d9fa3488cafa87f (DEFAULT_TABLE table=test-table [id=ff19ad66cd88468ebb955f890bbce69d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:22.739095 28269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f23ccf7f8b541c59d9fa3488cafa87f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:22.740013 28126 tablet_service.cc:1467] Processing CreateTablet for tablet 5f23ccf7f8b541c59d9fa3488cafa87f (DEFAULT_TABLE table=test-table [id=ff19ad66cd88468ebb955f890bbce69d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:22.740183 27979 tablet_service.cc:1467] Processing CreateTablet for tablet 5f23ccf7f8b541c59d9fa3488cafa87f (DEFAULT_TABLE table=test-table [id=ff19ad66cd88468ebb955f890bbce69d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:22.740338 28126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f23ccf7f8b541c59d9fa3488cafa87f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:22.740473 27979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f23ccf7f8b541c59d9fa3488cafa87f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:22.741904 28368 tablet_bootstrap.cc:492] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb: Bootstrap starting.
I20260808 14:39:22.742966 28368 tablet_bootstrap.cc:654] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:22.743489 28366 tablet_bootstrap.cc:492] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73: Bootstrap starting.
I20260808 14:39:22.744266 28368 tablet_bootstrap.cc:492] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb: No bootstrap required, opened a new log
I20260808 14:39:22.744385 28368 ts_tablet_manager.cc:1397] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:39:22.744412 28366 tablet_bootstrap.cc:654] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:22.744861 28368 raft_consensus.cc:348] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.744951 28367 tablet_bootstrap.cc:492] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588: Bootstrap starting.
I20260808 14:39:22.744995 28368 raft_consensus.cc:374] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:22.745035 28368 raft_consensus.cc:729] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43edeb6b22f84c4bb3629e52b7a6b4bb, State: Initialized, Role: FOLLOWER
I20260808 14:39:22.745162 28368 consensus_queue.cc:260] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.745466 28368 ts_tablet_manager.cc:1428] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:39:22.745923 28367 tablet_bootstrap.cc:654] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:22.747473 28367 tablet_bootstrap.cc:492] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588: No bootstrap required, opened a new log
I20260808 14:39:22.747570 28367 ts_tablet_manager.cc:1397] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:39:22.747954 28366 tablet_bootstrap.cc:492] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73: No bootstrap required, opened a new log
I20260808 14:39:22.748075 28366 ts_tablet_manager.cc:1397] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 14:39:22.748080 28367 raft_consensus.cc:348] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.748243 28367 raft_consensus.cc:374] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:22.748297 28367 raft_consensus.cc:729] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3572036761bb4f29b786328b3dcb7588, State: Initialized, Role: FOLLOWER
I20260808 14:39:22.748507 28367 consensus_queue.cc:260] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.748598 28366 raft_consensus.cc:348] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.748723 28366 raft_consensus.cc:374] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:22.748759 28366 raft_consensus.cc:729] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dab0325fbae4f069c67092426e7cc73, State: Initialized, Role: FOLLOWER
I20260808 14:39:22.748829 28367 ts_tablet_manager.cc:1428] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 14:39:22.748888 28366 consensus_queue.cc:260] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.749154 28366 ts_tablet_manager.cc:1428] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:39:22.754462 28414 raft_consensus.cc:482] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:39:22.754570 28414 raft_consensus.cc:504] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.755025 28414 leader_election.cc:288] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3572036761bb4f29b786328b3dcb7588 (127.19.144.65:37131), 3dab0325fbae4f069c67092426e7cc73 (127.19.144.66:38523)
I20260808 14:39:22.755401 28146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f23ccf7f8b541c59d9fa3488cafa87f" candidate_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dab0325fbae4f069c67092426e7cc73" is_pre_election: true
I20260808 14:39:22.755401 27999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f23ccf7f8b541c59d9fa3488cafa87f" candidate_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3572036761bb4f29b786328b3dcb7588" is_pre_election: true
I20260808 14:39:22.755548 28146 raft_consensus.cc:2454] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43edeb6b22f84c4bb3629e52b7a6b4bb in term 0.
I20260808 14:39:22.755548 27999 raft_consensus.cc:2454] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43edeb6b22f84c4bb3629e52b7a6b4bb in term 0.
I20260808 14:39:22.755811 28219 leader_election.cc:302] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dab0325fbae4f069c67092426e7cc73, 43edeb6b22f84c4bb3629e52b7a6b4bb; no voters:
I20260808 14:39:22.756074 28414 raft_consensus.cc:2781] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:39:22.756145 28414 raft_consensus.cc:482] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:39:22.756199 28414 raft_consensus.cc:3037] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:22.757292 28414 raft_consensus.cc:504] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.757764 28414 leader_election.cc:288] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [CANDIDATE]: Term 1 election: Requested vote from peers 3572036761bb4f29b786328b3dcb7588 (127.19.144.65:37131), 3dab0325fbae4f069c67092426e7cc73 (127.19.144.66:38523)
I20260808 14:39:22.758240 28146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f23ccf7f8b541c59d9fa3488cafa87f" candidate_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dab0325fbae4f069c67092426e7cc73"
I20260808 14:39:22.758253 27999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f23ccf7f8b541c59d9fa3488cafa87f" candidate_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3572036761bb4f29b786328b3dcb7588"
I20260808 14:39:22.758366 28146 raft_consensus.cc:3037] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:22.758391 27999 raft_consensus.cc:3037] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:22.759297 27999 raft_consensus.cc:2454] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43edeb6b22f84c4bb3629e52b7a6b4bb in term 1.
I20260808 14:39:22.759518 28146 raft_consensus.cc:2454] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43edeb6b22f84c4bb3629e52b7a6b4bb in term 1.
I20260808 14:39:22.759601 28222 leader_election.cc:302] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3572036761bb4f29b786328b3dcb7588, 43edeb6b22f84c4bb3629e52b7a6b4bb; no voters:
I20260808 14:39:22.759941 28414 raft_consensus.cc:2781] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:22.760066 28414 raft_consensus.cc:686] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 LEADER]: Becoming Leader. State: Replica: 43edeb6b22f84c4bb3629e52b7a6b4bb, State: Running, Role: LEADER
I20260808 14:39:22.760212 28414 consensus_queue.cc:237] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } }
I20260808 14:39:22.762115 27835 catalog_manager.cc:5495] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 43edeb6b22f84c4bb3629e52b7a6b4bb (127.19.144.67). New cstate: current_term: 1 leader_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43edeb6b22f84c4bb3629e52b7a6b4bb" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 40499 } health_report { overall_health: HEALTHY } } }
I20260808 14:39:22.776399 28365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.773336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44524 (local address 127.19.144.67:40499)
0808 14:39:22.773489 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:22.773493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:22.773505 (+ 12us) server_negotiation.cc:407] Connection header received
0808 14:39:22.773596 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:22.773598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:22.773708 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:22.773769 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:39:22.773967 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.774602 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.775514 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:22.775726 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:22.775807 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:22.776031 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:39:22.776103 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:39:22.776157 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:22.776236 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 14:39:22.776278 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:39:22.783061 28145 raft_consensus.cc:1261] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73 [term 1 FOLLOWER]: Refusing update from remote peer 43edeb6b22f84c4bb3629e52b7a6b4bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:22.783028 27999 raft_consensus.cc:1261] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Refusing update from remote peer 43edeb6b22f84c4bb3629e52b7a6b4bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:22.783551 28414 consensus_queue.cc:1035] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3572036761bb4f29b786328b3dcb7588" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 37131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:22.783756 28397 consensus_queue.cc:1035] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dab0325fbae4f069c67092426e7cc73" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 38523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:22.816834 27836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:22.816999 27836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:22.819150 27836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a265713a6614240ae6de7eb9f6bcf3d: Sending DeleteTablet for 3 replicas of tablet 5f23ccf7f8b541c59d9fa3488cafa87f
I20260808 14:39:22.819813 28269 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f23ccf7f8b541c59d9fa3488cafa87f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:22 UTC) from {username='oryx', principal='oryx/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:44500
I20260808 14:39:22.819854 28126 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f23ccf7f8b541c59d9fa3488cafa87f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:22 UTC) from {username='oryx', principal='oryx/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:34608
I20260808 14:39:22.819842 27979 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f23ccf7f8b541c59d9fa3488cafa87f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:22 UTC) from {username='oryx', principal='oryx/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:33150
I20260808 14:39:22.820034 28390 tablet_replica.cc:330] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73: stopping tablet replica
I20260808 14:39:22.820050 28392 tablet_replica.cc:330] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588: stopping tablet replica
I20260808 14:39:22.820180 28390 raft_consensus.cc:2229] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:22.820199 28392 raft_consensus.cc:2229] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:22.820324 28392 raft_consensus.cc:2258] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:22.820331 28390 raft_consensus.cc:2258] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:22.820546 28391 tablet_replica.cc:330] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb: stopping tablet replica
I20260808 14:39:22.820664 28391 raft_consensus.cc:2229] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:39:22.820873 28391 raft_consensus.cc:2258] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:22.821208 28392 ts_tablet_manager.cc:1905] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3572036761bb4f29b786328b3dcb7588: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:22.821369 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 27911
I20260808 14:39:22.823536 28391 ts_tablet_manager.cc:1905] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:22.825675 28391 ts_tablet_manager.cc:1918] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:39:22.825769 28391 log.cc:1196] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/5f23ccf7f8b541c59d9fa3488cafa87f
I20260808 14:39:22.826026 28391 ts_tablet_manager.cc:1939] T 5f23ccf7f8b541c59d9fa3488cafa87f P 43edeb6b22f84c4bb3629e52b7a6b4bb: Deleting consensus metadata
I20260808 14:39:22.826748 27823 catalog_manager.cc:4841] TS 43edeb6b22f84c4bb3629e52b7a6b4bb (127.19.144.67:40499): tablet 5f23ccf7f8b541c59d9fa3488cafa87f (table test-table [id=ff19ad66cd88468ebb955f890bbce69d]) successfully deleted
I20260808 14:39:22.830380 28390 ts_tablet_manager.cc:1905] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 14:39:22.832067 27823 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.144.65:37131 (error 108)
I20260808 14:39:22.832697 28390 ts_tablet_manager.cc:1918] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 14:39:22.832785 28390 log.cc:1196] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/5f23ccf7f8b541c59d9fa3488cafa87f
I20260808 14:39:22.833019 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 28058
I20260808 14:39:22.833102 28390 ts_tablet_manager.cc:1939] T 5f23ccf7f8b541c59d9fa3488cafa87f P 3dab0325fbae4f069c67092426e7cc73: Deleting consensus metadata
I20260808 14:39:22.833478 28450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.832883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:37131 (local address 127.0.0.1:33152)
0808 14:39:22.833248 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:22.833342 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:37131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":62,"threads_started":1}
W20260808 14:39:22.833858 27823 catalog_manager.cc:4584] TS 3572036761bb4f29b786328b3dcb7588 (127.19.144.65:37131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5f23ccf7f8b541c59d9fa3488cafa87f: Network error: Client connection negotiation failed: client connection to 127.19.144.65:37131: connect: Connection refused (error 111)
I20260808 14:39:22.843047 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 28202
I20260808 14:39:22.843839 28450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:22.843394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:38523 (local address 127.0.0.1:34628)
0808 14:39:22.843644 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:22.843713 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.66:38523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260808 14:39:22.853371 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 27803
2026-08-08T14:39:22Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3099 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:39:22 dist-test-slave-p3t5 krb5kdc[28453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:39:22 dist-test-slave-p3t5 krb5kdc[28453](info): set up 2 sockets
Aug 08 14:39:22 dist-test-slave-p3t5 krb5kdc[28453](info): commencing operation
krb5kdc: starting...
W20260808 14:39:23.940026 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:39:24 dist-test-slave-p3t5 krb5kdc[28453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:39:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:39:24.076726 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:38651
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:37399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:38651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:24.196200 28466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:24.196556 28466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:24.196640 28466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:24.200805 28466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:24.200949 28466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:24.201022 28466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:24.201067 28466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:24.201107 28466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:39:24.206657 28466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:37399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:38651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:38651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:24.207865 28466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:24.208994 28466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:24.217772 28475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:24.217772 28473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:24.217803 28472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:24.218098 28466 server_base.cc:1034] running on GCE node
I20260808 14:39:24.218770 28466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:24.220000 28466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:24.221199 28466 hybrid_clock.cc:648] HybridClock initialized: now 1786199964221165 us; error 48 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T14:39:24Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1379 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:39:24 dist-test-slave-p3t5 krb5kdc[28480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:39:24 dist-test-slave-p3t5 krb5kdc[28480](info): set up 2 sockets
Aug 08 14:39:24 dist-test-slave-p3t5 krb5kdc[28480](info): commencing operation
krb5kdc: starting...
W20260808 14:39:25.308220 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:39:25 dist-test-slave-p3t5 krb5kdc[28480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:39:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:39:25.452723 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:36269
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:36269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:25.575294 28493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:25.575608 28493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:25.575651 28493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:25.579707 28493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:25.579777 28493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:25.579797 28493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:25.579813 28493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:25.579828 28493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:39:25.585350 28493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:36269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:36269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:25.586637 28493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:25.587746 28493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:25.597275 28499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:25.597285 28502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:25.597281 28500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:25.597651 28493 server_base.cc:1034] running on GCE node
I20260808 14:39:25.598368 28493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:25.599560 28493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:25.600750 28493 hybrid_clock.cc:648] HybridClock initialized: now 1786199965600719 us; error 57 us; skew 500 ppm
Aug 08 14:39:25 dist-test-slave-p3t5 krb5kdc[28480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199965, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:25.611208 28493 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:39:25.612409 28493 webserver.cc:460] Webserver started at http://127.19.144.126:46259/ using document root <none> and password file <none>
I20260808 14:39:25.613078 28493 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:25.613150 28493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:25.613415 28493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:25.615816 28493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "a4701f7db3b7462cafe430697ab6331c"
format_stamp: "Formatted at 2026-08-08 14:39:25 on dist-test-slave-p3t5"
server_key: "543e57a488ec0286a7e27b964cc3ef7c"
server_key_iv: "144b0ac535383a3f68ac816221191eb8"
server_key_version: "encryptionkey@0"
I20260808 14:39:25.616466 28493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "a4701f7db3b7462cafe430697ab6331c"
format_stamp: "Formatted at 2026-08-08 14:39:25 on dist-test-slave-p3t5"
server_key: "543e57a488ec0286a7e27b964cc3ef7c"
server_key_iv: "144b0ac535383a3f68ac816221191eb8"
server_key_version: "encryptionkey@0"
I20260808 14:39:25.620748 28493 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 14:39:25.623642 28509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:25.624989 28493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:39:25.625136 28493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "a4701f7db3b7462cafe430697ab6331c"
format_stamp: "Formatted at 2026-08-08 14:39:25 on dist-test-slave-p3t5"
server_key: "543e57a488ec0286a7e27b964cc3ef7c"
server_key_iv: "144b0ac535383a3f68ac816221191eb8"
server_key_version: "encryptionkey@0"
I20260808 14:39:25.625267 28493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:25.644346 28493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:25.648295 28493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:25.648533 28493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:25.656291 28493 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:36269
I20260808 14:39:25.656313 28565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:36269 every 8 connection(s)
I20260808 14:39:25.657351 28493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:25.660435 28566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:25.661134 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 28493
I20260808 14:39:25.661291 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:39:25.661659 20033 external_mini_cluster.cc:1428] Setting key 7e147d8ea2c628ac8dc851bc66e9c556
I20260808 14:39:25.667583 28566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c: Bootstrap starting.
I20260808 14:39:25.670710 28566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c: Neither blocks nor log segments found. Creating new log.
Aug 08 14:39:25 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:25.671623 28566 log.cc:824] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:25.674531 28566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c: No bootstrap required, opened a new log
I20260808 14:39:25.677997 28569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:25.663135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41916 (local address 127.19.144.126:36269)
0808 14:39:25.663592 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:25.663612 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:25.663656 (+ 44us) server_negotiation.cc:407] Connection header received
0808 14:39:25.664354 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:25.664374 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:25.664733 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:25.665042 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:25.666064 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:25.667094 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:25.668245 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:25.668751 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:25.668876 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:25.672006 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:25.672015 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:25.672039 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:25.672081 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:25.674278 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:25.674816 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:25.674824 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:25.674831 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:25.675000 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:25.675367 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:25.675371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:25.675372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:25.675938 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:25.676156 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:25.676439 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 14:39:25.676796 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":151,"threads_started":1}
I20260808 14:39:25.679381 28566 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4701f7db3b7462cafe430697ab6331c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36269 } }
I20260808 14:39:25.679687 28566 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:25.679739 28566 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4701f7db3b7462cafe430697ab6331c, State: Initialized, Role: FOLLOWER
I20260808 14:39:25.680241 28566 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4701f7db3b7462cafe430697ab6331c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36269 } }
I20260808 14:39:25.680413 28566 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:25.680483 28566 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:25.680589 28566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:25.681691 28566 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4701f7db3b7462cafe430697ab6331c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36269 } }
I20260808 14:39:25.682076 28566 leader_election.cc:302] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4701f7db3b7462cafe430697ab6331c; no voters:
I20260808 14:39:25.682423 28566 leader_election.cc:288] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:39:25.682559 28571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:25.682821 28571 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 1 LEADER]: Becoming Leader. State: Replica: a4701f7db3b7462cafe430697ab6331c, State: Running, Role: LEADER
I20260808 14:39:25.683249 28571 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4701f7db3b7462cafe430697ab6331c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36269 } }
I20260808 14:39:25.683771 28566 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:25.685081 28573 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4701f7db3b7462cafe430697ab6331c. Latest consensus state: current_term: 1 leader_uuid: "a4701f7db3b7462cafe430697ab6331c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4701f7db3b7462cafe430697ab6331c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36269 } } }
I20260808 14:39:25.685215 28573 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:25.685652 28572 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4701f7db3b7462cafe430697ab6331c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4701f7db3b7462cafe430697ab6331c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36269 } } }
I20260808 14:39:25.685811 28572 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [sys.catalog]: This master's current role is: LEADER
W20260808 14:39:25.688848 28587 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:39:25.688922 28587 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:39:25.689009 28580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:25.689945 28580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:25.695498 28580 catalog_manager.cc:1347] Generated new cluster ID: 2925963a691f40908e5af587ba2a2184
I20260808 14:39:25.695590 28580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:25.707680 28580 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:39:25.709062 28580 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:25.722080 28580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c: Generated new TSK 0
I20260808 14:39:25.722951 28580 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:39:25.757975 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:36269
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:25.879274 28593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:25.879653 28593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:25.879746 28593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:25.883921 28593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:25.884042 28593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:25.884153 28593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:39:25.889873 28593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:36269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:25.891158 28593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:25.892310 28593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:25.902846 28601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:25.902946 28593 server_base.cc:1034] running on GCE node
W20260808 14:39:25.902870 28599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:25.902853 28598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:25.903631 28593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:25.904335 28593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:25.905530 28593 hybrid_clock.cc:648] HybridClock initialized: now 1786199965905514 us; error 29 us; skew 500 ppm
Aug 08 14:39:25 dist-test-slave-p3t5 krb5kdc[28480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199965, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:25.908512 28593 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:39:25.909451 28593 webserver.cc:460] Webserver started at http://127.19.144.65:43517/ using document root <none> and password file <none>
I20260808 14:39:25.910145 28593 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:25.910246 28593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:25.910495 28593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:25.912699 28593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "5cfa3a5451294fafad1730c8cbfd0f1a"
format_stamp: "Formatted at 2026-08-08 14:39:25 on dist-test-slave-p3t5"
server_key: "bf4d9b8dff05d285ed18ea1081339803"
server_key_iv: "3194f79fba1c1f3ebec8b774f290eeb7"
server_key_version: "encryptionkey@0"
I20260808 14:39:25.913326 28593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "5cfa3a5451294fafad1730c8cbfd0f1a"
format_stamp: "Formatted at 2026-08-08 14:39:25 on dist-test-slave-p3t5"
server_key: "bf4d9b8dff05d285ed18ea1081339803"
server_key_iv: "3194f79fba1c1f3ebec8b774f290eeb7"
server_key_version: "encryptionkey@0"
I20260808 14:39:25.917409 28593 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 14:39:25.920305 28608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:25.921712 28593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:25.921867 28593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "5cfa3a5451294fafad1730c8cbfd0f1a"
format_stamp: "Formatted at 2026-08-08 14:39:25 on dist-test-slave-p3t5"
server_key: "bf4d9b8dff05d285ed18ea1081339803"
server_key_iv: "3194f79fba1c1f3ebec8b774f290eeb7"
server_key_version: "encryptionkey@0"
I20260808 14:39:25.922008 28593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:25.938491 28593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:25.942190 28593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:25.942456 28593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:25.943182 28593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:25.944149 28593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:25.944242 28593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:25.944343 28593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:25.944396 28593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:25.953949 28593 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:39441
I20260808 14:39:25.953974 28726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:39441 every 8 connection(s)
I20260808 14:39:25.954959 28593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:39:25.956586 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 28593
I20260808 14:39:25.956732 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:39:25.957095 20033 external_mini_cluster.cc:1428] Setting key 9567b1a7d52ff8afc732c03aab19b229
Aug 08 14:39:25 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199965, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:39:25.970901 28569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:25.957195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:49461 (local address 127.19.144.126:36269)
0808 14:39:25.957379 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:25.957385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:25.958733 (+ 1348us) server_negotiation.cc:407] Connection header received
0808 14:39:25.959708 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:25.959712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:25.959765 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:25.959846 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:25.961122 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:25.961753 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:25.962963 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:25.963204 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:25.963325 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:25.966568 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:25.966574 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:25.966577 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:25.966616 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:25.968534 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:25.969113 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:25.969117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:25.969119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:25.969218 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:25.969676 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:25.969683 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:25.969689 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:25.969959 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:25.970052 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:25.970628 (+ 576us) server_negotiation.cc:299] Negotiation successful
0808 14:39:25.970753 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 14:39:25.971989 28729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:25.957651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:49461)
0808 14:39:25.958548 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:25.958612 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:25.959489 (+ 877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:25.960047 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:25.960054 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:25.960450 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:25.960965 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:25.960980 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:25.961902 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:25.961905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:25.962815 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:25.962825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:25.963373 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:25.963378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:25.963567 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:25.964322 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:25.964342 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:25.966343 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:25.968687 (+ 2344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:25.968693 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:25.968707 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:25.968994 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:25.969344 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:25.969347 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:25.969348 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:25.969529 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:25.970064 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:25.970069 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:25.970345 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 14:39:25.970520 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 14:39:25.970969 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":169,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
I20260808 14:39:25.973502 28727 heartbeater.cc:347] Connected to a master server at 127.19.144.126:36269
I20260808 14:39:25.973850 28727 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:25.974494 28727 heartbeater.cc:511] Master 127.19.144.126:36269 requested a full tablet report, sending...
I20260808 14:39:25.976531 28527 ts_manager.cc:194] Registered new tserver with Master: 5cfa3a5451294fafad1730c8cbfd0f1a (127.19.144.65:39441)
I20260808 14:39:25.977803 28527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:49461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:39:25.992620 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:36269
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:26.112640 28732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:26.112933 28732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:26.112979 28732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:26.116994 28732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:26.117063 28732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:26.117125 28732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:39:26.122542 28732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:36269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:26.123665 28732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:26.124702 28732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:26.135589 28739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:26.135600 28742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:26.135581 28738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:26.135931 28732 server_base.cc:1034] running on GCE node
I20260808 14:39:26.136395 28732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:26.137149 28732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:26.138376 28732 hybrid_clock.cc:648] HybridClock initialized: now 1786199966138332 us; error 55 us; skew 500 ppm
Aug 08 14:39:26 dist-test-slave-p3t5 krb5kdc[28480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199966, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:26.141427 28732 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:39:26.142403 28732 webserver.cc:460] Webserver started at http://127.19.144.66:41799/ using document root <none> and password file <none>
I20260808 14:39:26.142952 28732 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:26.143014 28732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:26.143184 28732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:26.145228 28732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "73070216c45b47dbbaf5204636893de9"
format_stamp: "Formatted at 2026-08-08 14:39:26 on dist-test-slave-p3t5"
server_key: "876ff0637e9662112133945ff023bf7f"
server_key_iv: "676e7a9cd93795866b087bd8b4d516de"
server_key_version: "encryptionkey@0"
I20260808 14:39:26.145824 28732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "73070216c45b47dbbaf5204636893de9"
format_stamp: "Formatted at 2026-08-08 14:39:26 on dist-test-slave-p3t5"
server_key: "876ff0637e9662112133945ff023bf7f"
server_key_iv: "676e7a9cd93795866b087bd8b4d516de"
server_key_version: "encryptionkey@0"
I20260808 14:39:26.149843 28732 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:39:26.152513 28749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:26.154114 28732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:26.154244 28732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "73070216c45b47dbbaf5204636893de9"
format_stamp: "Formatted at 2026-08-08 14:39:26 on dist-test-slave-p3t5"
server_key: "876ff0637e9662112133945ff023bf7f"
server_key_iv: "676e7a9cd93795866b087bd8b4d516de"
server_key_version: "encryptionkey@0"
I20260808 14:39:26.154362 28732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:26.172175 28732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:26.175494 28732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:26.175716 28732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:26.176352 28732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:26.177301 28732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:26.177361 28732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:26.177425 28732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:26.177460 28732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:26.187373 28732 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:44445
I20260808 14:39:26.187454 28866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:44445 every 8 connection(s)
I20260808 14:39:26.188369 28732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:39:26.190944 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 28732
I20260808 14:39:26.191105 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:39:26.191412 20033 external_mini_cluster.cc:1428] Setting key ad45da4954bc483b0b19be75da099555
Aug 08 14:39:26 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199966, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
I20260808 14:39:26.204406 28569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:26.190457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:46511 (local address 127.19.144.126:36269)
0808 14:39:26.190602 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:26.190606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:26.191752 (+ 1146us) server_negotiation.cc:407] Connection header received
0808 14:39:26.192794 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:26.192798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:26.192851 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:26.192948 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:26.194733 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:26.195303 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:26.196591 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:26.196825 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:26.196946 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:26.200225 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:26.200228 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:26.200230 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:26.200259 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:26.201948 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:26.202626 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:26.202628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:26.202629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:26.202708 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:26.203143 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:26.203145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:26.203146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:26.203342 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:26.203418 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:26.204147 (+ 729us) server_negotiation.cc:299] Negotiation successful
0808 14:39:26.204263 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:39:26.205446 28869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:26.190825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:46511)
0808 14:39:26.191572 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:26.191628 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:26.192545 (+ 917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:26.193146 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:26.193155 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:26.193766 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:26.194539 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:26.194564 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:26.195471 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:26.195475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:26.196442 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:26.196452 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:26.196998 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:26.197002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:26.197248 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:26.197990 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:26.198019 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:26.199996 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:26.202080 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:26.202089 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:26.202110 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:26.202508 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:26.202807 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:26.202810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:26.202814 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:26.203034 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:26.203459 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:26.203466 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:26.203844 (+ 378us) client_negotiation.cc:769] Sending connection context
0808 14:39:26.204107 (+ 263us) client_negotiation.cc:240] Negotiation successful
0808 14:39:26.204451 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":141,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
I20260808 14:39:26.207126 28867 heartbeater.cc:347] Connected to a master server at 127.19.144.126:36269
I20260808 14:39:26.207422 28867 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:26.207996 28867 heartbeater.cc:511] Master 127.19.144.126:36269 requested a full tablet report, sending...
I20260808 14:39:26.209331 28527 ts_manager.cc:194] Registered new tserver with Master: 73070216c45b47dbbaf5204636893de9 (127.19.144.66:44445)
I20260808 14:39:26.210227 28527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:46511
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:39:26.223145 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:36269
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:26.342643 28872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:26.342996 28872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:26.343083 28872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:26.347383 28872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:26.347486 28872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:26.347576 28872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:39:26.353235 28872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:36269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:26.354529 28872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:26.355661 28872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:39:26.366257 28872 server_base.cc:1034] running on GCE node
W20260808 14:39:26.366209 28882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:26.366209 28879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:26.366212 28880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:26.366907 28872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:26.367611 28872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:26.368775 28872 hybrid_clock.cc:648] HybridClock initialized: now 1786199966368759 us; error 29 us; skew 500 ppm
Aug 08 14:39:26 dist-test-slave-p3t5 krb5kdc[28480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199966, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:26.371937 28872 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:39:26.372855 28872 webserver.cc:460] Webserver started at http://127.19.144.67:45527/ using document root <none> and password file <none>
I20260808 14:39:26.373488 28872 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:26.373555 28872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:26.373809 28872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:26.375982 28872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "3242c18f94ec4da0b70be722956c8226"
format_stamp: "Formatted at 2026-08-08 14:39:26 on dist-test-slave-p3t5"
server_key: "e2bd01d3b1007f23f0aadc2620140739"
server_key_iv: "b15d9a4100f71a9c99227cce4617bab0"
server_key_version: "encryptionkey@0"
I20260808 14:39:26.376534 28872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "3242c18f94ec4da0b70be722956c8226"
format_stamp: "Formatted at 2026-08-08 14:39:26 on dist-test-slave-p3t5"
server_key: "e2bd01d3b1007f23f0aadc2620140739"
server_key_iv: "b15d9a4100f71a9c99227cce4617bab0"
server_key_version: "encryptionkey@0"
I20260808 14:39:26.380602 28872 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 14:39:26.383363 28889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:26.384974 28872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:39:26.385105 28872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "3242c18f94ec4da0b70be722956c8226"
format_stamp: "Formatted at 2026-08-08 14:39:26 on dist-test-slave-p3t5"
server_key: "e2bd01d3b1007f23f0aadc2620140739"
server_key_iv: "b15d9a4100f71a9c99227cce4617bab0"
server_key_version: "encryptionkey@0"
I20260808 14:39:26.385226 28872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:26.418013 28872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:26.421694 28872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:26.421916 28872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:26.422596 28872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:26.423846 28872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:26.423911 28872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:26.423992 28872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:26.424031 28872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:26.433688 28872 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:41617
I20260808 14:39:26.433728 29005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:41617 every 8 connection(s)
I20260808 14:39:26.434701 28872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:39:26.441788 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 28872
I20260808 14:39:26.441938 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:39:26.442211 20033 external_mini_cluster.cc:1428] Setting key c8972bf99b2a5509da80f60c0a3e2d13
Aug 08 14:39:26 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199966, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:26.450878 28569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:26.436884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:59915 (local address 127.19.144.126:36269)
0808 14:39:26.437042 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:26.437047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:26.438683 (+ 1636us) server_negotiation.cc:407] Connection header received
0808 14:39:26.439969 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:26.439989 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:26.440066 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:26.440162 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:26.441438 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:26.442201 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:26.443881 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:26.444186 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:26.444317 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:26.447081 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:26.447084 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:26.447087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:26.447115 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:26.448736 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:26.449381 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:26.449385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:26.449387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:26.449485 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:26.449871 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:26.449874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:26.449875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:26.450063 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:26.450142 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:26.450639 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 14:39:26.450749 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 14:39:26.451793 29009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:26.437497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:59915)
0808 14:39:26.438487 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:26.438552 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:26.439677 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:26.440336 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:26.440343 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:26.440731 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:26.441268 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:26.441283 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:26.442366 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:26.442369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:26.443758 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:26.443769 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:26.444349 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:26.444354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:26.444555 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:26.445204 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:26.445222 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:26.446889 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:26.448862 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:26.448867 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:26.448901 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:26.449282 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:26.449575 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:26.449577 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:26.449578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:26.449785 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:26.450155 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:26.450160 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:26.450416 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 14:39:26.450577 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 14:39:26.450856 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":163,"threads_started":1}
I20260808 14:39:26.453063 29006 heartbeater.cc:347] Connected to a master server at 127.19.144.126:36269
I20260808 14:39:26.453354 29006 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:26.453903 29006 heartbeater.cc:511] Master 127.19.144.126:36269 requested a full tablet report, sending...
I20260808 14:39:26.455004 28527 ts_manager.cc:194] Registered new tserver with Master: 3242c18f94ec4da0b70be722956c8226 (127.19.144.67:41617)
I20260808 14:39:26.455587 28527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:59915
I20260808 14:39:26.456445 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.65.keytab.
Entry for principal oryx/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.65.keytab.
WARNING: no policy specified for oryx/127.19.144.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.144.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.66.keytab.
Entry for principal oryx/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.66.keytab.
WARNING: no policy specified for oryx/127.19.144.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.144.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.67.keytab.
Entry for principal oryx/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.67.keytab.
I20260808 14:39:26.541181 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 28593
I20260808 14:39:26.548931 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 28732
I20260808 14:39:26.555997 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 28872
I20260808 14:39:26.565413 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 28493
I20260808 14:39:26.574008 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:36269
--webserver_interface=127.19.144.126
--webserver_port=46259
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:36269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:26.694348 29016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:26.694653 29016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:26.694693 29016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:26.698863 29016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:26.698930 29016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:26.698951 29016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:26.698968 29016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:26.699021 29016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:39:26.705101 29016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:36269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:36269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:26.706427 29016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:26.707471 29016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:26.716866 29024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:26.716883 29021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:26.716866 29022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:26.716950 29016 server_base.cc:1034] running on GCE node
I20260808 14:39:26.717770 29016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:26.718976 29016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:26.720184 29016 hybrid_clock.cc:648] HybridClock initialized: now 1786199966720176 us; error 33 us; skew 500 ppm
Aug 08 14:39:26 dist-test-slave-p3t5 krb5kdc[28480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199966, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:26.723254 29016 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:39:26.724339 29016 webserver.cc:460] Webserver started at http://127.19.144.126:46259/ using document root <none> and password file <none>
I20260808 14:39:26.724956 29016 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:26.725082 29016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:26.729220 29016 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:39:26.731676 29031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:26.733129 29016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260808 14:39:26.733314 29016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "a4701f7db3b7462cafe430697ab6331c"
format_stamp: "Formatted at 2026-08-08 14:39:25 on dist-test-slave-p3t5"
server_key: "543e57a488ec0286a7e27b964cc3ef7c"
server_key_iv: "144b0ac535383a3f68ac816221191eb8"
server_key_version: "encryptionkey@0"
I20260808 14:39:26.733975 29016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:26.753820 29016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:26.757452 29016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:26.757702 29016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:26.765663 29016 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:36269
I20260808 14:39:26.765679 29085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:36269 every 8 connection(s)
I20260808 14:39:26.766873 29016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:26.771361 29086 sys_catalog.cc:263] Verifying existing consensus state
I20260808 14:39:26.771857 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 29016
I20260808 14:39:26.772397 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:39441
--local_ip_for_outbound_sockets=127.19.144.65
--tserver_master_addrs=127.19.144.126:36269
--webserver_port=43517
--webserver_interface=127.19.144.65
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.144.65
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:39:26.773849 29086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c: Bootstrap starting.
I20260808 14:39:26.791415 29086 log.cc:824] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:26.798834 29086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:39:26.799260 29086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c: Bootstrap complete.
I20260808 14:39:26.803699 29086 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4701f7db3b7462cafe430697ab6331c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36269 } }
I20260808 14:39:26.804167 29086 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4701f7db3b7462cafe430697ab6331c, State: Initialized, Role: FOLLOWER
I20260808 14:39:26.804731 29086 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4701f7db3b7462cafe430697ab6331c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36269 } }
I20260808 14:39:26.804888 29086 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:26.804980 29086 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:26.805104 29086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 1 FOLLOWER]: Advancing to term 2
I20260808 14:39:26.807248 29086 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4701f7db3b7462cafe430697ab6331c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36269 } }
I20260808 14:39:26.807662 29086 leader_election.cc:302] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4701f7db3b7462cafe430697ab6331c; no voters:
I20260808 14:39:26.808022 29086 leader_election.cc:288] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 14:39:26.808182 29090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 2 FOLLOWER]: Leader election won for term 2
I20260808 14:39:26.808444 29090 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [term 2 LEADER]: Becoming Leader. State: Replica: a4701f7db3b7462cafe430697ab6331c, State: Running, Role: LEADER
I20260808 14:39:26.808792 29090 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4701f7db3b7462cafe430697ab6331c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36269 } }
I20260808 14:39:26.809336 29086 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:26.809909 29090 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4701f7db3b7462cafe430697ab6331c. Latest consensus state: current_term: 2 leader_uuid: "a4701f7db3b7462cafe430697ab6331c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4701f7db3b7462cafe430697ab6331c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36269 } } }
I20260808 14:39:26.810039 29090 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:26.810498 29094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:26.810451 29091 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a4701f7db3b7462cafe430697ab6331c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4701f7db3b7462cafe430697ab6331c" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36269 } } }
I20260808 14:39:26.810698 29091 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:26.814029 29094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:26.814795 29094 catalog_manager.cc:1259] Loaded cluster ID: 2925963a691f40908e5af587ba2a2184
I20260808 14:39:26.814859 29094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:26.817297 29094 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:26.818033 29094 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a4701f7db3b7462cafe430697ab6331c: Loaded TSK: 0
I20260808 14:39:26.819262 29094 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 14:39:26.897970 29088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:26.898250 29088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:26.898299 29088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:26.902652 29088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:26.902724 29088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:26.902825 29088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:39:26.908293 29088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.65.keytab
--principal=oryx/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:39441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=43517
--tserver_master_addrs=127.19.144.126:36269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:26.909472 29088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:26.910749 29088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:26.922633 29112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:26.922650 29115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:26.922662 29113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:26.922879 29088 server_base.cc:1034] running on GCE node
I20260808 14:39:26.923483 29088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:26.924345 29088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:26.925567 29088 hybrid_clock.cc:648] HybridClock initialized: now 1786199966925524 us; error 52 us; skew 500 ppm
Aug 08 14:39:26 dist-test-slave-p3t5 krb5kdc[28480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199966, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:26.929301 29088 init.cc:377] Logged in from keytab as oryx/127.19.144.65@KRBTEST.COM (short username oryx)
I20260808 14:39:26.930429 29088 webserver.cc:460] Webserver started at http://127.19.144.65:43517/ using document root <none> and password file <none>
I20260808 14:39:26.931054 29088 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:26.931178 29088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:26.935497 29088 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 14:39:26.938016 29122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:26.939647 29088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:39:26.939774 29088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "5cfa3a5451294fafad1730c8cbfd0f1a"
format_stamp: "Formatted at 2026-08-08 14:39:25 on dist-test-slave-p3t5"
server_key: "bf4d9b8dff05d285ed18ea1081339803"
server_key_iv: "3194f79fba1c1f3ebec8b774f290eeb7"
server_key_version: "encryptionkey@0"
I20260808 14:39:26.940518 29088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:26.966526 29088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:26.969997 29088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:26.970171 29088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:26.970758 29088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:26.971604 29088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:26.971654 29088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:26.971697 29088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:26.971716 29088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:26.981132 29088 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:39441
I20260808 14:39:26.981169 29238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:39441 every 8 connection(s)
I20260808 14:39:26.982312 29088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:39:26.991058 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 29088
I20260808 14:39:26.991564 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:44445
--local_ip_for_outbound_sockets=127.19.144.66
--tserver_master_addrs=127.19.144.126:36269
--webserver_port=41799
--webserver_interface=127.19.144.66
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.144.66
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:39:26 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:26 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:26.996170 29242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:26.984689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:50361)
0808 14:39:26.985397 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:26.985440 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:26.986328 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:26.987388 (+ 1060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:26.987395 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:26.987766 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:26.988251 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:26.988265 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:26.989551 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:26.989553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:26.990469 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:26.990480 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:26.991528 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:26.991531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:26.991737 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:26.992448 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:26.992465 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:26.995289 (+ 2824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":157,"threads_started":1}
W20260808 14:39:26.996572 29241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:26.984541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:50361 (local address 127.19.144.126:36269)
0808 14:39:26.985030 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:26.985052 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:26.985575 (+ 523us) server_negotiation.cc:407] Connection header received
0808 14:39:26.986701 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:26.986712 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:26.987017 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:26.987286 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:26.988424 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:26.989420 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:26.990599 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:26.991158 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:26.991325 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:26.995626 (+ 4301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:50361: BlockingRecv error: recv got EOF from 127.19.144.65:50361 (error 108)
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":107,"threads_started":1}
W20260808 14:39:26.997181 29239 heartbeater.cc:644] Failed to heartbeat to 127.19.144.126:36269 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.144.126:36269: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Aug 08 14:39:26 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:26 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:27.004294 29242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:26.997821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:48561)
0808 14:39:26.997972 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:26.997991 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:26.998099 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:26.998390 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:26.998395 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:26.998616 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:26.998787 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:26.998797 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:26.999572 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:26.999579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:27.000627 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:27.000643 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.001132 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.001138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:27.001272 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.001893 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:27.001916 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:27.004122 (+ 2206us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 14:39:27.004395 29241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:26.997897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:48561 (local address 127.19.144.126:36269)
0808 14:39:26.998023 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:26.998026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:26.998041 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:39:26.998204 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:26.998206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:26.998250 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:26.998321 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:26.998923 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:26.999452 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.000776 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.001001 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.001087 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.004271 (+ 3184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:48561: BlockingRecv error: recv got EOF from 127.19.144.65:48561 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 14:39:27 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:27 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:27.010378 29242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:27.004978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:56461)
0808 14:39:27.005099 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:27.005117 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:27.005229 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:27.005485 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:27.005488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:27.005720 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:27.005802 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:27.005807 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.006554 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.006559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:27.007103 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:27.007111 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.007581 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.007585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:27.007668 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.008198 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:27.008211 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:27.010226 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 14:39:27.010557 29241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:27.005121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:56461 (local address 127.19.144.126:36269)
0808 14:39:27.005271 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:27.005277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:27.005294 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:39:27.005340 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:27.005343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:27.005379 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:27.005468 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:27.005928 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.006441 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.007229 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.007454 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.007566 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.010428 (+ 2862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:56461: BlockingRecv error: recv got EOF from 127.19.144.65:56461 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 14:39:27.010608 29239 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 14:39:27.114392 29243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:27.114738 29243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:27.114817 29243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:27.119269 29243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:27.119365 29243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:27.119462 29243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:39:27.125129 29243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.66.keytab
--principal=oryx/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:44445
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=41799
--tserver_master_addrs=127.19.144.126:36269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:27.126446 29243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:27.127612 29243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:27.137972 29248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:27.137938 29251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:27.137949 29249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:27.138216 29243 server_base.cc:1034] running on GCE node
I20260808 14:39:27.138726 29243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:27.139447 29243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:27.140643 29243 hybrid_clock.cc:648] HybridClock initialized: now 1786199967140611 us; error 52 us; skew 500 ppm
Aug 08 14:39:27 dist-test-slave-p3t5 krb5kdc[28480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199967, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:27.143739 29243 init.cc:377] Logged in from keytab as oryx/127.19.144.66@KRBTEST.COM (short username oryx)
I20260808 14:39:27.144711 29243 webserver.cc:460] Webserver started at http://127.19.144.66:41799/ using document root <none> and password file <none>
I20260808 14:39:27.145314 29243 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:27.145443 29243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:27.149706 29243 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 14:39:27.152004 29260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:27.153272 29243 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 14:39:27.153429 29243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "73070216c45b47dbbaf5204636893de9"
format_stamp: "Formatted at 2026-08-08 14:39:26 on dist-test-slave-p3t5"
server_key: "876ff0637e9662112133945ff023bf7f"
server_key_iv: "676e7a9cd93795866b087bd8b4d516de"
server_key_version: "encryptionkey@0"
I20260808 14:39:27.154094 29243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:27.169515 29243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:27.172919 29243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:27.173144 29243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:27.173870 29243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:27.174769 29243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:27.174842 29243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:27.174919 29243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:27.174964 29243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:27.184829 29243 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:44445
I20260808 14:39:27.184857 29380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:44445 every 8 connection(s)
I20260808 14:39:27.185990 29243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:39:27.191041 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 29243
I20260808 14:39:27.191519 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:41617
--local_ip_for_outbound_sockets=127.19.144.67
--tserver_master_addrs=127.19.144.126:36269
--webserver_port=45527
--webserver_interface=127.19.144.67
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.144.67
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:39:27 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:27 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 14:39:27.199103 29241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:27.188127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:42371 (local address 127.19.144.126:36269)
0808 14:39:27.188306 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:27.188310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:27.189504 (+ 1194us) server_negotiation.cc:407] Connection header received
0808 14:39:27.190728 (+ 1224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:27.190732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:27.190800 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:27.190914 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:27.192322 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.192931 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.194286 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.194520 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.194639 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.198992 (+ 4353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:42371: BlockingRecv error: recv got EOF from 127.19.144.66:42371 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 14:39:27.200110 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:27.188564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:42371)
0808 14:39:27.189323 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:27.189369 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:27.190479 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:27.191110 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:27.191118 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:27.191571 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:27.192158 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:27.192174 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.193067 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.193073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:27.194148 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:27.194159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.194684 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.194689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:27.194934 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.195620 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:27.195646 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:27.198858 (+ 3212us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":120,"threads_started":1}
W20260808 14:39:27.200663 29381 heartbeater.cc:644] Failed to heartbeat to 127.19.144.126:36269 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.144.126:36269: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Aug 08 14:39:27 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:27 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:27.206493 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:27.201127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:50419)
0808 14:39:27.201248 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:27.201266 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:27.201385 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:27.201674 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:27.201677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:27.201855 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:27.201957 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:27.201962 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.202696 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.202700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:27.203269 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:27.203277 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.203733 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.203737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:27.203831 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.204354 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:27.204367 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:27.206338 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 14:39:27.206620 29241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:27.201282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:50419 (local address 127.19.144.126:36269)
0808 14:39:27.201438 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:27.201442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:27.201457 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:39:27.201509 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:27.201511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:27.201547 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:27.201651 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:27.202073 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.202584 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.203394 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.203617 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.203729 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.206483 (+ 2754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:50419: BlockingRecv error: recv got EOF from 127.19.144.66:50419 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 14:39:27 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:27 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:27.212939 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:27.207270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:41257)
0808 14:39:27.207399 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:27.207414 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:27.207497 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:27.207732 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:27.207735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:27.207915 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:27.208015 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:27.208021 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.208716 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.208719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:27.209605 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:27.209647 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.210078 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.210082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:27.210172 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.210687 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:27.210703 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:27.212773 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 14:39:27.213037 29241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:27.207386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:41257 (local address 127.19.144.126:36269)
0808 14:39:27.207497 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:27.207501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:27.207513 (+ 12us) server_negotiation.cc:407] Connection header received
0808 14:39:27.207586 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:27.207589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:27.207625 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:27.207689 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:27.208125 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.208615 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.209758 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.209974 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.210053 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.212911 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:41257: BlockingRecv error: recv got EOF from 127.19.144.66:41257 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 14:39:27.213351 29381 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 14:39:27.316022 29384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:27.316330 29384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:27.316381 29384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:27.320601 29384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:27.320670 29384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:27.320734 29384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:39:27.326363 29384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/krb5kdc/oryx_127.19.144.67.keytab
--principal=oryx/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:41617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=45527
--tserver_master_addrs=127.19.144.126:36269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:27.327499 29384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:27.328531 29384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:27.338711 29390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:27.338739 29392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:27.338739 29389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:27.338795 29384 server_base.cc:1034] running on GCE node
I20260808 14:39:27.339432 29384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:27.340165 29384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:27.341331 29384 hybrid_clock.cc:648] HybridClock initialized: now 1786199967341316 us; error 30 us; skew 500 ppm
Aug 08 14:39:27 dist-test-slave-p3t5 krb5kdc[28480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199967, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:27.344442 29384 init.cc:377] Logged in from keytab as oryx/127.19.144.67@KRBTEST.COM (short username oryx)
I20260808 14:39:27.345505 29384 webserver.cc:460] Webserver started at http://127.19.144.67:45527/ using document root <none> and password file <none>
I20260808 14:39:27.346133 29384 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:27.346213 29384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:27.350526 29384 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:39:27.352881 29399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:27.354274 29384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:27.354437 29384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "3242c18f94ec4da0b70be722956c8226"
format_stamp: "Formatted at 2026-08-08 14:39:26 on dist-test-slave-p3t5"
server_key: "e2bd01d3b1007f23f0aadc2620140739"
server_key_iv: "b15d9a4100f71a9c99227cce4617bab0"
server_key_version: "encryptionkey@0"
I20260808 14:39:27.355012 29384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:27.371416 29384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:27.374910 29384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:27.375141 29384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:27.375802 29384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:27.376713 29384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:27.376791 29384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:27.376889 29384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:27.376942 29384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:27.386519 29384 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:41617
I20260808 14:39:27.386544 29516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:41617 every 8 connection(s)
I20260808 14:39:27.387621 29384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:39:27.390745 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 29384
Aug 08 14:39:27 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:27 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 14:39:27.400861 29241 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:27.389764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:60613 (local address 127.19.144.126:36269)
0808 14:39:27.389932 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:27.389936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:27.391320 (+ 1384us) server_negotiation.cc:407] Connection header received
0808 14:39:27.392500 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:27.392506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:27.392563 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:27.392655 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:27.394104 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.394920 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.396347 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.396560 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.396671 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.400714 (+ 4043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:60613: BlockingRecv error: recv got EOF from 127.19.144.67:60613 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 14:39:27.401700 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:27.390232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:60613)
0808 14:39:27.391099 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:27.391170 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:27.392264 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:27.392851 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:27.392860 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:27.393258 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:27.393949 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:27.393968 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.395051 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.395054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:27.396216 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:27.396226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.396723 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.396727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:27.396908 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.397499 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:27.397520 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:27.400590 (+ 3070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":200,"threads_started":1}
W20260808 14:39:27.402256 29517 heartbeater.cc:644] Failed to heartbeat to 127.19.144.126:36269 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.144.126:36269: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
I20260808 14:39:27.402282 29521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:27.391826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41922 (local address 127.19.144.126:36269)
0808 14:39:27.392122 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:27.392126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:27.392143 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:39:27.392190 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:27.392194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:27.392242 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:27.392363 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:27.393301 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.394099 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.395286 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.395614 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.395727 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.397356 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:27.397365 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:27.397389 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:27.397429 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:27.399607 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:27.400209 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:27.400214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:27.400220 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:27.400377 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:27.400707 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:27.400711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:27.400712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:27.401277 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:27.401440 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:27.401854 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 14:39:27.402124 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":115,"threads_started":1}
Aug 08 14:39:27 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:27 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:27.409153 29522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:27.403456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:48797)
0808 14:39:27.403800 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:27.403812 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:27.403943 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:27.404198 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:27.404201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:27.404395 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:27.404491 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:27.404496 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.405415 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.405419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:27.406092 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:27.406102 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.406593 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.406597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:27.406690 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.407203 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:27.407215 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:27.408972 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":134,"threads_started":1}
W20260808 14:39:27.409211 29521 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:27.403543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:48797 (local address 127.19.144.126:36269)
0808 14:39:27.403667 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:27.403672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:27.403896 (+ 224us) server_negotiation.cc:407] Connection header received
0808 14:39:27.404045 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:27.404049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:27.404095 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:27.404179 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:27.404610 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.405300 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.406220 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.406458 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.406579 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.409097 (+ 2518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:48797: BlockingRecv error: recv got EOF from 127.19.144.67:48797 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 14:39:27 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:27 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:27.415045 29522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:27.409836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:49347)
0808 14:39:27.409959 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:27.409992 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:27.410075 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:27.410383 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:27.410386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:27.410546 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:27.410638 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:27.410643 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.411421 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.411424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:27.411966 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:27.411974 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.412420 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.412424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:27.412530 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.413021 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:27.413033 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:27.414940 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 14:39:27.415235 29521 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:27.409947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:49347 (local address 127.19.144.126:36269)
0808 14:39:27.410109 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:27.410115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:27.410131 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:39:27.410167 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:27.410172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:27.410254 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:27.410366 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:27.410753 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.411308 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.412089 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:27.412305 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:27.412420 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:27.415068 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:49347: BlockingRecv error: recv got EOF from 127.19.144.67:49347 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260808 14:39:27.415339 29517 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 14:39:28 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:28 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:28.017889 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:28.011404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:50421)
0808 14:39:28.011704 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:28.011719 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:28.011844 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:28.012127 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:28.012129 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:28.012345 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:28.012477 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:28.012483 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:28.013259 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:28.013263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:28.013942 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:28.013954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:28.014445 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:28.014449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:28.014543 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:28.015145 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:28.015159 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:28.017656 (+ 2497us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":91,"threads_started":1}
W20260808 14:39:28.017921 29524 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:28.011498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:50421 (local address 127.19.144.126:36269)
0808 14:39:28.011786 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:28.011789 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:28.011805 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:39:28.011944 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:28.011947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:28.011993 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:28.012083 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:28.012591 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:28.013129 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:28.014077 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:28.014305 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:28.014400 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:28.017796 (+ 3396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:50421: BlockingRecv error: recv got EOF from 127.19.144.65:50421 (error 108)
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":115,"threads_started":1}
Aug 08 14:39:28 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:28 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:28.220754 29525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:28.214161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:48213)
0808 14:39:28.214504 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:28.214521 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:28.214683 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:28.215013 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:28.215016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:28.215264 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:28.215400 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:28.215406 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:28.216207 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:28.216212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:28.216822 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:28.216830 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:28.217311 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:28.217314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:28.217422 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:28.218018 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:28.218031 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:28.220600 (+ 2569us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":94,"threads_started":1}
W20260808 14:39:28.221021 29524 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:28.214268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:48213 (local address 127.19.144.126:36269)
0808 14:39:28.214472 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:28.214477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:28.214620 (+ 143us) server_negotiation.cc:407] Connection header received
0808 14:39:28.214836 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:28.214838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:28.214886 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:28.214983 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:28.215540 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:28.216085 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:28.216955 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:28.217186 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:28.217304 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:28.220783 (+ 3479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:48213: BlockingRecv error: recv got EOF from 127.19.144.66:48213 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 08 14:39:28 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:28 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:28.421811 29526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:28.416068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:52289)
0808 14:39:28.416386 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:28.416400 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:28.416504 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:28.416743 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:28.416746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:28.416948 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:28.417082 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:28.417087 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:28.417895 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:28.417898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:28.418525 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:28.418535 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:28.418978 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:28.418981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:28.419054 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:28.419611 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:28.419623 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:28.421671 (+ 2048us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":91,"threads_started":1}
W20260808 14:39:28.422044 29524 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:28.416143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:52289 (local address 127.19.144.126:36269)
0808 14:39:28.416283 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:28.416287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:28.416469 (+ 182us) server_negotiation.cc:407] Connection header received
0808 14:39:28.416578 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:28.416581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:28.416628 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:28.416696 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:28.417191 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:28.417750 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:28.418632 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:28.418876 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:28.418953 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:28.421895 (+ 2942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:52289: BlockingRecv error: recv got EOF from 127.19.144.67:52289 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 14:39:29 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:29 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:29.024959 29527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:29.018874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:37109)
0808 14:39:29.019222 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:29.019238 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:29.019346 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:29.019654 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:29.019657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:29.019885 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:29.019989 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:29.019995 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:29.021049 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:29.021055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:29.021710 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:29.021722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:29.022171 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:29.022174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:29.022259 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:29.022787 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:29.022799 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:29.024822 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":99,"threads_started":1}
W20260808 14:39:29.025173 29528 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:29.019041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:37109 (local address 127.19.144.126:36269)
0808 14:39:29.019321 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:29.019326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:29.019357 (+ 31us) server_negotiation.cc:407] Connection header received
0808 14:39:29.019418 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:29.019421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:29.019475 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:29.019588 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:29.020116 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:29.020909 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:29.021842 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:29.022065 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:29.022144 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:29.024985 (+ 2841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:37109: BlockingRecv error: recv got EOF from 127.19.144.65:37109 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":63,"threads_started":1}
Aug 08 14:39:29 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:29 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:29.228014 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:29.221721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:44205)
0808 14:39:29.222067 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:29.222080 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:29.222213 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:29.222579 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:29.222582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:29.222858 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:29.222982 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:29.222988 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:29.223937 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:29.223940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:29.224529 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:29.224537 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:29.225033 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:29.225036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:29.225110 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:29.225776 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:29.225789 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:29.227830 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":123,"threads_started":1}
W20260808 14:39:29.228169 29528 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:29.221864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:44205 (local address 127.19.144.126:36269)
0808 14:39:29.222055 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:29.222060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:29.222142 (+ 82us) server_negotiation.cc:407] Connection header received
0808 14:39:29.222371 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:29.222375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:29.222427 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:29.222513 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:29.223114 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:29.223809 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:29.224671 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:29.224911 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:29.225005 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:29.228011 (+ 3006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:44205: BlockingRecv error: recv got EOF from 127.19.144.66:44205 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 08 14:39:29 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:29 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:29.428920 29530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:29.422718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:59629)
0808 14:39:29.423185 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:29.423203 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:29.423381 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:29.423677 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:29.423680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:29.423919 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:29.424020 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:29.424026 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:29.424822 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:29.424828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:29.425412 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:29.425420 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:29.425963 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:29.425968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:29.426058 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:29.426617 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:29.426630 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:29.428762 (+ 2132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":109,"threads_started":1}
W20260808 14:39:29.429121 29528 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:29.422951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:59629 (local address 127.19.144.126:36269)
0808 14:39:29.423202 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:29.423208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:29.423302 (+ 94us) server_negotiation.cc:407] Connection header received
0808 14:39:29.423494 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:29.423497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:29.423553 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:29.423649 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:29.424151 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:29.424698 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:29.425540 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:29.425841 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:29.425956 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:29.428954 (+ 2998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:59629: BlockingRecv error: recv got EOF from 127.19.144.67:59629 (error 108)
Metrics: {"server-negotiator.queue_time_us":166}
Aug 08 14:39:30 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:30 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:30.031659 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:30.025816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:43597)
0808 14:39:30.026131 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:30.026145 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:30.026243 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:30.026471 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:30.026473 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:30.026657 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:30.026773 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:30.026778 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:30.027529 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:30.027531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:30.028110 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:30.028118 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:30.028593 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:30.028597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:30.028668 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:30.029310 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:30.029323 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:30.031522 (+ 2199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":96,"threads_started":1}
W20260808 14:39:30.031780 29532 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:30.025936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:43597 (local address 127.19.144.126:36269)
0808 14:39:30.026200 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:30.026204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:30.026217 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:39:30.026332 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:30.026335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:30.026379 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:30.026457 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:30.026877 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:30.027428 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:30.028218 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:30.028448 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:30.028528 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:30.031669 (+ 3141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:43597: BlockingRecv error: recv got EOF from 127.19.144.65:43597 (error 108)
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":90,"threads_started":1}
Aug 08 14:39:30 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:30 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:30.235008 29533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:30.229049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:54671)
0808 14:39:30.229338 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:30.229353 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:30.229466 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:30.229796 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:30.229799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:30.230052 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:30.230163 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:30.230170 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:30.231043 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:30.231048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:30.231654 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:30.231685 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:30.232156 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:30.232159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:30.232251 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:30.232847 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:30.232859 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:30.234867 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":97,"threads_started":1}
W20260808 14:39:30.235194 29532 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:30.229142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:54671 (local address 127.19.144.126:36269)
0808 14:39:30.229313 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:30.229317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:30.229429 (+ 112us) server_negotiation.cc:407] Connection header received
0808 14:39:30.229556 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:30.229558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:30.229606 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:30.229703 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:30.230300 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:30.230917 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:30.231802 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:30.232037 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:30.232151 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:30.235025 (+ 2874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:54671: BlockingRecv error: recv got EOF from 127.19.144.66:54671 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 08 14:39:30 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:30 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:30.436112 29534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:30.429909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:42549)
0808 14:39:30.430240 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:30.430257 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:30.430447 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:30.430739 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:30.430742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:30.430978 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:30.431078 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:30.431084 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:30.431866 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:30.431871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:30.432454 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:30.432462 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:30.432938 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:30.432942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:30.433029 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:30.433645 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:30.433658 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:30.435936 (+ 2278us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":106,"threads_started":1}
W20260808 14:39:30.436265 29532 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:30.429969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:42549 (local address 127.19.144.126:36269)
0808 14:39:30.430210 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:30.430215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:30.430359 (+ 144us) server_negotiation.cc:407] Connection header received
0808 14:39:30.430560 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:30.430564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:30.430621 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:30.430713 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:30.431210 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:30.431745 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:30.432581 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:30.432809 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:30.432933 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:30.436147 (+ 3214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:42549: BlockingRecv error: recv got EOF from 127.19.144.67:42549 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
Aug 08 14:39:31 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:31 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:31.045735 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:31.032716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:54355)
0808 14:39:31.033062 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:31.033077 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:31.033181 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:31.033487 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:31.033490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:31.033730 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:31.033875 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:31.033882 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:31.034701 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:31.034704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:31.035297 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:31.035307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:31.035769 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:31.035773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:31.035855 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:31.043262 (+ 7407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:31.043278 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:31.045540 (+ 2262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":111,"threads_started":1}
W20260808 14:39:31.046052 29537 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:31.032833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:54355 (local address 127.19.144.126:36269)
0808 14:39:31.033131 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:31.033134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:31.033150 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:39:31.033290 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:31.033293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:31.033341 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:31.033423 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:31.034017 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:31.034572 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:31.035416 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:31.035642 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:31.035721 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:31.045749 (+ 10028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:54355: BlockingRecv error: recv got EOF from 127.19.144.65:54355 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":128,"threads_started":1}
Aug 08 14:39:31 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:31 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:31.242172 29538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:31.235849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:55637)
0808 14:39:31.236223 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:31.236238 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:31.236368 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:31.236651 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:31.236653 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:31.236883 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:31.236992 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:31.236998 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:31.237979 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:31.237982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:31.238602 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:31.238611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:31.239118 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:31.239121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:31.239189 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:31.239772 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:31.239784 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:31.242043 (+ 2259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":144,"threads_started":1}
W20260808 14:39:31.242379 29537 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:31.235925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:55637 (local address 127.19.144.126:36269)
0808 14:39:31.236098 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:31.236102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:31.236342 (+ 240us) server_negotiation.cc:407] Connection header received
0808 14:39:31.236466 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:31.236469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:31.236529 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:31.236594 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:31.237141 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:31.237860 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:31.238741 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:31.239004 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:31.239096 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:31.242196 (+ 3100us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:55637: BlockingRecv error: recv got EOF from 127.19.144.66:55637 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 14:39:31 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:31 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:31.446013 29539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:31.437031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:54681)
0808 14:39:31.437401 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:31.437414 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:31.437556 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:31.437883 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:31.437887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:31.438150 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:31.438276 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:31.438283 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:31.439113 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:31.439118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:31.439740 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:31.439751 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:31.440242 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:31.440246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:31.440338 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:31.443522 (+ 3184us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:31.443537 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:31.445807 (+ 2270us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":126,"threads_started":1}
W20260808 14:39:31.446216 29537 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:31.437092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:54681 (local address 127.19.144.126:36269)
0808 14:39:31.437249 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:31.437254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:31.437501 (+ 247us) server_negotiation.cc:407] Connection header received
0808 14:39:31.437699 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:31.437702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:31.437767 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:31.437861 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:31.438408 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:31.438994 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:31.439874 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:31.440109 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:31.440225 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:31.446075 (+ 5850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:54681: BlockingRecv error: recv got EOF from 127.19.144.67:54681 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 14:39:32 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:32 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:32.052637 29540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:32.046758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:51187)
0808 14:39:32.047109 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:32.047128 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:32.047247 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:32.047507 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:32.047510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:32.047711 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:32.047814 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:32.047820 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:32.048585 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:32.048588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:32.049179 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:32.049188 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:32.049739 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:32.049746 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:32.049851 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:32.050406 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:32.050421 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:32.052515 (+ 2094us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":134,"threads_started":1}
W20260808 14:39:32.052846 29541 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:32.046874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:51187 (local address 127.19.144.126:36269)
0808 14:39:32.047133 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:32.047136 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:32.047212 (+ 76us) server_negotiation.cc:407] Connection header received
0808 14:39:32.047332 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:32.047334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:32.047386 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:32.047469 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:32.047925 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:32.048476 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:32.049298 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:32.049527 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:32.049605 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:32.052660 (+ 3055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:51187: BlockingRecv error: recv got EOF from 127.19.144.65:51187 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":91,"threads_started":1}
Aug 08 14:39:32 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:32 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:32.249228 29542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:32.243157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:43755)
0808 14:39:32.243500 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:32.243518 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:32.243639 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:32.243955 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:32.243959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:32.244187 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:32.244293 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:32.244299 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:32.245116 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:32.245121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:32.245824 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:32.245834 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:32.246336 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:32.246340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:32.246429 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:32.246994 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:32.247006 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:32.249100 (+ 2094us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":147,"threads_started":1}
W20260808 14:39:32.249434 29541 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:32.243275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:43755 (local address 127.19.144.126:36269)
0808 14:39:32.243433 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:32.243439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:32.243639 (+ 200us) server_negotiation.cc:407] Connection header received
0808 14:39:32.243718 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:32.243722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:32.243782 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:32.243887 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:32.244449 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:32.244995 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:32.245964 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:32.246196 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:32.246331 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:32.249292 (+ 2961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:43755: BlockingRecv error: recv got EOF from 127.19.144.66:43755 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 14:39:32 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:32 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:32.454021 29543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:32.447339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:59489)
0808 14:39:32.447769 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:32.447787 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:32.447933 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:32.448278 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:32.448281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:32.448575 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:32.448694 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:32.448700 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:32.449810 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:32.449813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:32.450419 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:32.450428 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:32.450887 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:32.450890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:32.450965 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:32.451574 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:32.451587 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:32.453822 (+ 2235us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":97,"threads_started":1}
W20260808 14:39:32.454070 29541 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:32.447412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:59489 (local address 127.19.144.126:36269)
0808 14:39:32.447771 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:32.447778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:32.447894 (+ 116us) server_negotiation.cc:407] Connection header received
0808 14:39:32.448069 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:32.448073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:32.448133 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:32.448228 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:32.448869 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:32.449653 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:32.450544 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:32.450771 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:32.450861 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:32.453964 (+ 3103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:59489: BlockingRecv error: recv got EOF from 127.19.144.67:59489 (error 108)
Metrics: {"server-negotiator.queue_time_us":266}
Aug 08 14:39:33 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:33 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:33.059686 29544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:33.053531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:58931)
0808 14:39:33.053948 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:33.053963 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:33.054078 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:33.054345 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:33.054347 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:33.054556 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:33.054657 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:33.054662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:33.055438 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:33.055441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:33.056021 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:33.056031 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:33.056535 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:33.056537 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:33.056604 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:33.057227 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:33.057239 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:33.059504 (+ 2265us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":117,"threads_started":1}
W20260808 14:39:33.059770 29545 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:33.053709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:58931 (local address 127.19.144.126:36269)
0808 14:39:33.053954 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:33.053958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:33.054047 (+ 89us) server_negotiation.cc:407] Connection header received
0808 14:39:33.054162 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:33.054164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:33.054211 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:33.054292 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:33.054761 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:33.055316 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:33.056177 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:33.056417 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:33.056509 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:33.059636 (+ 3127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:58931: BlockingRecv error: recv got EOF from 127.19.144.65:58931 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":65,"threads_started":1}
Aug 08 14:39:33 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:33 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:33.256165 29546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:33.250180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:40083)
0808 14:39:33.250543 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:33.250561 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:33.250720 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:33.251048 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:33.251052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:33.251258 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:33.251369 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:33.251375 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:33.252371 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:33.252377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:33.252997 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:33.253007 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:33.253441 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:33.253443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:33.253508 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:33.254120 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:33.254141 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:33.256045 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":98,"threads_started":1}
W20260808 14:39:33.256351 29545 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:33.250331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:40083 (local address 127.19.144.126:36269)
0808 14:39:33.250527 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:33.250533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:33.250653 (+ 120us) server_negotiation.cc:407] Connection header received
0808 14:39:33.250844 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:33.250847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:33.250900 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:33.250970 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:33.251486 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:33.252243 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:33.253117 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:33.253338 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:33.253432 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:33.256186 (+ 2754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:40083: BlockingRecv error: recv got EOF from 127.19.144.66:40083 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 14:39:33 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:33 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:33.461452 29547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:33.455154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:34661)
0808 14:39:33.455496 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:33.455514 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:33.455665 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:33.456092 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:33.456095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:33.456400 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:33.456538 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:33.456548 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:33.457436 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:33.457439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:33.458149 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:33.458159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:33.458598 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:33.458600 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:33.458663 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:33.459391 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:33.459405 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:33.461318 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":103,"threads_started":1}
W20260808 14:39:33.461664 29545 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:33.455358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:34661 (local address 127.19.144.126:36269)
0808 14:39:33.455540 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:33.455545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:33.455591 (+ 46us) server_negotiation.cc:407] Connection header received
0808 14:39:33.455901 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:33.455906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:33.455972 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:33.456060 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:33.456743 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:33.457324 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:33.458259 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:33.458490 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:33.458570 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:33.461476 (+ 2906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:34661: BlockingRecv error: recv got EOF from 127.19.144.67:34661 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 14:39:34 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:34 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:34.066305 29548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:34.060548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:55327)
0808 14:39:34.060868 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:34.060888 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:34.060981 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:34.061239 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:34.061241 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:34.061436 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:34.061540 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:34.061545 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:34.062433 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:34.062435 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:34.062997 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:34.063005 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:34.063481 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:34.063484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:34.063552 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:34.064109 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:34.064121 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:34.066180 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":95,"threads_started":1}
W20260808 14:39:34.066530 29549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:34.060654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:55327 (local address 127.19.144.126:36269)
0808 14:39:34.060917 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:34.060921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:34.060935 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:39:34.061073 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:34.061076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:34.061121 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:34.061204 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:34.061733 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:34.062325 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:34.063125 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:34.063366 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:34.063451 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:34.066368 (+ 2917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:55327: BlockingRecv error: recv got EOF from 127.19.144.65:55327 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":79,"threads_started":1}
Aug 08 14:39:34 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:34 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:34.263641 29550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:34.257062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:41523)
0808 14:39:34.257436 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:34.257452 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:34.257677 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:34.257987 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:34.257989 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:34.258191 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:34.258306 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:34.258312 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:34.259237 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:34.259239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:34.259914 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:34.259925 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:34.260426 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:34.260429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:34.260497 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:34.261171 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:34.261184 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:34.263459 (+ 2275us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":81,"threads_started":1}
W20260808 14:39:34.263731 29549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:34.257289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:41523 (local address 127.19.144.126:36269)
0808 14:39:34.257500 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:34.257504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:34.257592 (+ 88us) server_negotiation.cc:407] Connection header received
0808 14:39:34.257781 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:34.257784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:34.257833 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:34.257926 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:34.258465 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:34.259133 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:34.260044 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:34.260318 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:34.260406 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:34.263611 (+ 3205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:41523: BlockingRecv error: recv got EOF from 127.19.144.66:41523 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 08 14:39:34 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:34 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:34.468888 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:34.462352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:56165)
0808 14:39:34.462833 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:34.462846 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:34.463002 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:34.463294 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:34.463297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:34.463526 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:34.463623 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:34.463629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:34.464440 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:34.464445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:34.465019 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:34.465027 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:34.465510 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:34.465514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:34.465600 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:34.466258 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:34.466277 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:34.468763 (+ 2486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":117,"threads_started":1}
W20260808 14:39:34.469055 29549 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:34.462501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:56165 (local address 127.19.144.126:36269)
0808 14:39:34.462737 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:34.462742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:34.462938 (+ 196us) server_negotiation.cc:407] Connection header received
0808 14:39:34.463111 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:34.463114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:34.463170 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:34.463269 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:34.463756 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:34.464324 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:34.465144 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:34.465384 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:34.465493 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:34.468898 (+ 3405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:56165: BlockingRecv error: recv got EOF from 127.19.144.67:56165 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Aug 08 14:39:35 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:35 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:35.074059 29552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:35.067296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:44161)
0808 14:39:35.067821 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:35.067842 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:35.067988 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:35.068303 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:35.068305 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:35.068504 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:35.068610 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:35.068616 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:35.069483 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:35.069486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:35.070137 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:35.070148 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:35.070683 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:35.070686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:35.070781 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:35.071452 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:35.071480 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:35.073872 (+ 2392us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":120,"threads_started":1}
W20260808 14:39:35.074154 29553 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:35.067558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:44161 (local address 127.19.144.126:36269)
0808 14:39:35.067950 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:35.067953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:35.067969 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:39:35.068101 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:35.068103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:35.068152 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:35.068254 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:35.068769 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:35.069345 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:35.070281 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:35.070523 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:35.070632 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:35.074019 (+ 3387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:44161: BlockingRecv error: recv got EOF from 127.19.144.65:44161 (error 108)
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":78,"threads_started":1}
Aug 08 14:39:35 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:35 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:35.270905 29554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:35.264620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:56729)
0808 14:39:35.264985 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:35.264999 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:35.265140 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:35.265432 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:35.265435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:35.265693 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:35.265796 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:35.265802 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:35.266748 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:35.266753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:35.267328 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:35.267336 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:35.267808 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:35.267811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:35.267901 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:35.268577 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:35.268590 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:35.270762 (+ 2172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":107,"threads_started":1}
W20260808 14:39:35.271217 29553 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:35.264725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:56729 (local address 127.19.144.126:36269)
0808 14:39:35.264905 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:35.264910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:35.265085 (+ 175us) server_negotiation.cc:407] Connection header received
0808 14:39:35.265251 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:35.265254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:35.265307 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:35.265399 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:35.265947 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:35.266621 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:35.267458 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:35.267689 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:35.267804 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:35.270947 (+ 3143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:56729: BlockingRecv error: recv got EOF from 127.19.144.66:56729 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 08 14:39:35 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:35 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:35.475826 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:35.469915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:50839)
0808 14:39:35.470220 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:35.470238 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:35.470404 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:35.470705 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:35.470708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:35.470930 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:35.471032 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:35.471038 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:35.471829 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:35.471834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:35.472446 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:35.472455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:35.472941 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:35.472945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:35.473034 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:35.473666 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:35.473686 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:35.475694 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":104,"threads_started":1}
W20260808 14:39:35.476125 29553 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:35.469970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:50839 (local address 127.19.144.126:36269)
0808 14:39:35.470112 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:35.470117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:35.470328 (+ 211us) server_negotiation.cc:407] Connection header received
0808 14:39:35.470513 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:35.470518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:35.470580 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:35.470672 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:35.471158 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:35.471706 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:35.472576 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:35.472804 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:35.472934 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:35.475847 (+ 2913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:50839: BlockingRecv error: recv got EOF from 127.19.144.67:50839 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 14:39:36 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:36 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:36.085860 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:36.075141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:50497)
0808 14:39:36.075456 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:36.075470 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:36.075576 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:36.075847 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:36.075849 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:36.076040 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:36.076141 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:36.076146 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:36.076928 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:36.076931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:36.077517 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:36.077532 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:36.078075 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:36.078077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:36.078146 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:36.083420 (+ 5274us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:36.083440 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:36.085724 (+ 2284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":108,"threads_started":1}
W20260808 14:39:36.086068 29557 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:36.075220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:50497 (local address 127.19.144.126:36269)
0808 14:39:36.075490 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:36.075494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:36.075527 (+ 33us) server_negotiation.cc:407] Connection header received
0808 14:39:36.075670 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:36.075672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:36.075720 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:36.075798 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:36.076250 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:36.076790 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:36.077656 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:36.077955 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:36.078039 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:36.085853 (+ 7814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:50497: BlockingRecv error: recv got EOF from 127.19.144.65:50497 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":65,"threads_started":1}
Aug 08 14:39:36 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:36 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:36.278353 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:36.271850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:33803)
0808 14:39:36.272234 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:36.272251 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:36.272431 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:36.272740 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:36.272743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:36.272991 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:36.273098 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:36.273104 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:36.273984 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:36.273990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:36.274747 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:36.274760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:36.275232 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:36.275236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:36.275339 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:36.276002 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:36.276015 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:36.278158 (+ 2143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":148,"threads_started":1}
W20260808 14:39:36.278419 29557 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:36.272044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:33803 (local address 127.19.144.126:36269)
0808 14:39:36.272230 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:36.272236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:36.272364 (+ 128us) server_negotiation.cc:407] Connection header received
0808 14:39:36.272547 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:36.272550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:36.272609 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:36.272706 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:36.273238 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:36.273865 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:36.274876 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:36.275105 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:36.275216 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:36.278306 (+ 3090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:33803: BlockingRecv error: recv got EOF from 127.19.144.66:33803 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 14:39:36 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:36 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:36.482616 29560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:36.476709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:44867)
0808 14:39:36.477009 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:36.477023 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:36.477130 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:36.477400 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:36.477402 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:36.477599 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:36.477765 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:36.477772 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:36.478494 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:36.478497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:36.479059 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:36.479066 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:36.479475 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:36.479478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:36.479537 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:36.480102 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:36.480115 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:36.482442 (+ 2327us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":105,"threads_started":1}
W20260808 14:39:36.482863 29557 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:36.476800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:44867 (local address 127.19.144.126:36269)
0808 14:39:36.476962 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:36.476969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:36.477099 (+ 130us) server_negotiation.cc:407] Connection header received
0808 14:39:36.477216 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:36.477218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:36.477269 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:36.477339 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:36.477878 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:36.478400 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:36.479162 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:36.479377 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:36.479454 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:36.482741 (+ 3287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:44867: BlockingRecv error: recv got EOF from 127.19.144.67:44867 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 14:39:37 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:37 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:37.092216 29561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:37.086700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:47879)
0808 14:39:37.087022 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:37.087036 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:37.087126 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:37.087406 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:37.087408 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:37.087605 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:37.087709 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:37.087714 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:37.088496 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:37.088499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:37.089078 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:37.089086 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:37.089500 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:37.089502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:37.089566 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:37.090120 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:37.090133 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:37.092091 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":102,"threads_started":1}
W20260808 14:39:37.092501 29562 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:37.086823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:47879 (local address 127.19.144.126:36269)
0808 14:39:37.087073 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:37.087076 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:37.087090 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:39:37.087215 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:37.087218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:37.087271 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:37.087365 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:37.087848 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:37.088388 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:37.089183 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:37.089402 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:37.089479 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:37.092348 (+ 2869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:47879: BlockingRecv error: recv got EOF from 127.19.144.65:47879 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":91,"threads_started":1}
Aug 08 14:39:37 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:37 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:37.285336 29563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:37.279310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:36827)
0808 14:39:37.279698 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:37.279713 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:37.279859 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:37.280163 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:37.280166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:37.280409 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:37.280507 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:37.280513 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:37.281334 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:37.281339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:37.281982 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:37.281991 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:37.282473 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:37.282476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:37.282566 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:37.283153 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:37.283166 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:37.285154 (+ 1988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":112,"threads_started":1}
W20260808 14:39:37.285365 29562 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:37.279467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:36827 (local address 127.19.144.126:36269)
0808 14:39:37.279666 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:37.279671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:37.279805 (+ 134us) server_negotiation.cc:407] Connection header received
0808 14:39:37.279990 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:37.279993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:37.280040 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:37.280132 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:37.280635 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:37.281214 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:37.282117 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:37.282348 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:37.282456 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:37.285267 (+ 2811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:36827: BlockingRecv error: recv got EOF from 127.19.144.66:36827 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 08 14:39:37 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:37 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:37.491047 29564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:37.483869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:48801)
0808 14:39:37.484223 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:37.484240 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:37.484397 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:37.484691 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:37.484694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:37.484933 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:37.485028 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:37.485033 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:37.486015 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:37.486021 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:37.487025 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:37.487038 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:37.487722 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:37.487726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:37.487821 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:37.488512 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:37.488527 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:37.490929 (+ 2402us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":131,"threads_started":1}
W20260808 14:39:37.491446 29562 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:37.484058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:48801 (local address 127.19.144.126:36269)
0808 14:39:37.484224 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:37.484230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:37.484332 (+ 102us) server_negotiation.cc:407] Connection header received
0808 14:39:37.484513 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:37.484516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:37.484572 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:37.484663 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:37.485158 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:37.485854 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:37.487179 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:37.487565 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:37.487680 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:37.491314 (+ 3634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:48801: BlockingRecv error: recv got EOF from 127.19.144.67:48801 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 08 14:39:38 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:38 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:38.099992 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:38.093172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:38879)
0808 14:39:38.093543 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:38.093559 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:38.093794 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:38.094134 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:38.094137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:38.094340 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:38.094443 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:38.094449 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:38.095387 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:38.095391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:38.096252 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:38.096264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:38.096925 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:38.096928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:38.096994 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:38.097604 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:38.097651 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:38.099842 (+ 2191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":102,"threads_started":1}
W20260808 14:39:38.100123 29566 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:38.093348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:38879 (local address 127.19.144.126:36269)
0808 14:39:38.093584 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:38.093588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:38.093632 (+ 44us) server_negotiation.cc:407] Connection header received
0808 14:39:38.093919 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:38.093921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:38.093972 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:38.094057 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:38.094602 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:38.095252 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:38.096415 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:38.096813 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:38.096902 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:38.099985 (+ 3083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:38879: BlockingRecv error: recv got EOF from 127.19.144.65:38879 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":65,"threads_started":1}
Aug 08 14:39:38 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:38 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:38.292722 29567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:38.286333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:50805)
0808 14:39:38.286627 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:38.286651 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:38.286779 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:38.287077 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:38.287080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:38.287283 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:38.287487 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:38.287497 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:38.288503 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:38.288509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:38.289336 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:38.289348 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:38.289802 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:38.289805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:38.289875 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:38.290424 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:38.290435 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:38.292588 (+ 2153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":103,"threads_started":1}
W20260808 14:39:38.292889 29566 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:38.286461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:50805 (local address 127.19.144.126:36269)
0808 14:39:38.286659 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:38.286663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:38.286713 (+ 50us) server_negotiation.cc:407] Connection header received
0808 14:39:38.286876 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:38.286896 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:38.286949 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:38.287037 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:38.287632 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:38.288374 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:38.289456 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:38.289708 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:38.289792 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:38.292741 (+ 2949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:50805: BlockingRecv error: recv got EOF from 127.19.144.66:50805 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 08 14:39:38 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:38 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:38.498448 29568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:38.492006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:37963)
0808 14:39:38.492501 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:38.492519 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:38.492650 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:38.493128 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:38.493132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:38.493403 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:38.493568 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:38.493578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:38.494619 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:38.494622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:38.495215 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:38.495224 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:38.495644 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:38.495646 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:38.495710 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:38.496285 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:38.496310 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:38.498325 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":86,"threads_started":1}
W20260808 14:39:38.498638 29566 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:38.492061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:37963 (local address 127.19.144.126:36269)
0808 14:39:38.492196 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:38.492201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:38.492592 (+ 391us) server_negotiation.cc:407] Connection header received
0808 14:39:38.492815 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:38.492819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:38.492890 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:38.492975 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:38.493716 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:38.494507 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:38.495325 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:38.495545 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:38.495623 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:38.498471 (+ 2848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:37963: BlockingRecv error: recv got EOF from 127.19.144.67:37963 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 14:39:39 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:39 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:39.106659 29569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:39.100850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:54979)
0808 14:39:39.101147 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:39.101160 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:39.101259 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:39.101499 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:39.101501 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:39.101718 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:39.101819 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:39.101825 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:39.102665 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:39.102667 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:39.103219 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:39.103227 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:39.103701 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:39.103703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:39.103767 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:39.104353 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:39.104366 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:39.106481 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":101,"threads_started":1}
W20260808 14:39:39.107098 29570 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:39.101000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:54979 (local address 127.19.144.126:36269)
0808 14:39:39.101261 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:39.101265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:39.101281 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:39:39.101327 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:39.101329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:39.101374 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:39.101465 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:39.101958 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:39.102561 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:39.103344 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:39.103593 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:39.103692 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:39.106642 (+ 2950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:54979: BlockingRecv error: recv got EOF from 127.19.144.65:54979 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":94,"threads_started":1}
Aug 08 14:39:39 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:39 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:39.299875 29571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:39.293822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:40845)
0808 14:39:39.294219 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:39.294233 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:39.294400 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:39.294703 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:39.294706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:39.294951 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:39.295055 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:39.295061 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:39.295920 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:39.295924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:39.296496 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:39.296505 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:39.296988 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:39.296992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:39.297086 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:39.297729 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:39.297742 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:39.299685 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":81,"threads_started":1}
W20260808 14:39:39.299980 29570 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:39.293979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:40845 (local address 127.19.144.126:36269)
0808 14:39:39.294131 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:39.294135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:39.294318 (+ 183us) server_negotiation.cc:407] Connection header received
0808 14:39:39.294517 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:39.294521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:39.294580 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:39.294678 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:39.295191 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:39.295744 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:39.296627 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:39.296862 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:39.296981 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:39.299851 (+ 2870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:40845: BlockingRecv error: recv got EOF from 127.19.144.66:40845 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 14:39:39 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:39 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:39.505113 29572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:39.499240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:45769)
0808 14:39:39.499596 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:39.499612 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:39.499764 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:39.500057 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:39.500060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:39.500301 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:39.500402 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:39.500408 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:39.501208 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:39.501212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:39.501875 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:39.501884 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:39.502373 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:39.502377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:39.502465 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:39.502987 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:39.502999 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:39.504988 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":112,"threads_started":1}
W20260808 14:39:39.505281 29570 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:39.499363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:45769 (local address 127.19.144.126:36269)
0808 14:39:39.499560 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:39.499565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:39.499700 (+ 135us) server_negotiation.cc:407] Connection header received
0808 14:39:39.499881 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:39.499884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:39.499944 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:39.500029 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:39.500532 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:39.501087 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:39.502001 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:39.502248 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:39.502367 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:39.505148 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:45769: BlockingRecv error: recv got EOF from 127.19.144.67:45769 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 08 14:39:40 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:40 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:40.114045 29574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:40.107751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:56091)
0808 14:39:40.108058 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:40.108076 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:40.108211 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:40.108563 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:40.108569 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:40.108856 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:40.109042 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:40.109054 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:40.109979 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:40.109985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:40.110711 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:40.110728 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:40.111173 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:40.111175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:40.111247 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:40.111824 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:40.111837 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:40.113918 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":105,"threads_started":1}
W20260808 14:39:40.114254 29573 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:40.107773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:56091 (local address 127.19.144.126:36269)
0808 14:39:40.108078 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:40.108082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:40.108171 (+ 89us) server_negotiation.cc:407] Connection header received
0808 14:39:40.108325 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:40.108328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:40.108390 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:40.108503 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:40.109201 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:40.109835 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:40.110838 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:40.111065 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:40.111147 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:40.114082 (+ 2935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:56091: BlockingRecv error: recv got EOF from 127.19.144.65:56091 (error 108)
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":108,"threads_started":1}
Aug 08 14:39:40 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:40 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:40.307331 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:40.301086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:49035)
0808 14:39:40.301400 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:40.301449 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:40.301574 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:40.301951 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:40.301954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:40.302250 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:40.302358 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:40.302364 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:40.303401 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:40.303405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:40.303991 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:40.304001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:40.304424 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:40.304427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:40.304498 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:40.305152 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:40.305165 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:40.307179 (+ 2014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":116,"threads_started":1}
W20260808 14:39:40.307710 29573 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:40.301341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:49035 (local address 127.19.144.126:36269)
0808 14:39:40.301502 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:40.301506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:40.301522 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:39:40.301711 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:40.301716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:40.301813 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:40.301921 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:40.302524 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:40.303285 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:40.304103 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:40.304316 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:40.304408 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:40.307389 (+ 2981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:49035: BlockingRecv error: recv got EOF from 127.19.144.66:49035 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 14:39:40 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:40 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:40.512864 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:40.506204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:50033)
0808 14:39:40.506486 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:40.506502 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:40.506627 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:40.506925 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:40.506931 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:40.507176 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:40.507328 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:40.507337 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:40.508389 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:40.508392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:40.509092 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:40.509104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:40.509591 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:40.509593 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:40.509768 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:40.510490 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:40.510503 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:40.512682 (+ 2179us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":99,"threads_started":1}
W20260808 14:39:40.512982 29573 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:40.506270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:50033 (local address 127.19.144.126:36269)
0808 14:39:40.506443 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:40.506447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:40.506601 (+ 154us) server_negotiation.cc:407] Connection header received
0808 14:39:40.506710 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:40.506715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:40.506788 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:40.506867 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:40.507473 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:40.508255 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:40.509239 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:40.509481 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:40.509561 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:40.512826 (+ 3265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:50033: BlockingRecv error: recv got EOF from 127.19.144.67:50033 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 14:39:41 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:41 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:41.126286 29578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:41.114954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:35729)
0808 14:39:41.115367 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:41.115391 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:41.115525 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:41.115921 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:41.115924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:41.116149 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:41.116256 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:41.116262 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:41.117150 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:41.117154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:41.117871 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:41.117884 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:41.118422 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:41.118426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:41.118508 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:41.123854 (+ 5346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:41.123870 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:41.126098 (+ 2228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":93,"threads_started":1}
W20260808 14:39:41.126430 29579 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:41.115045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:35729 (local address 127.19.144.126:36269)
0808 14:39:41.115399 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:41.115403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:41.115467 (+ 64us) server_negotiation.cc:407] Connection header received
0808 14:39:41.115639 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:41.115643 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:41.115709 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:41.115829 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:41.116381 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:41.116999 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:41.118019 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:41.118262 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:41.118366 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:41.126265 (+ 7899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:35729: BlockingRecv error: recv got EOF from 127.19.144.65:35729 (error 108)
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":67,"threads_started":1}
Aug 08 14:39:41 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:41 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:41.314917 29580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:41.308260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:60615)
0808 14:39:41.308580 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:41.308598 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:41.308774 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:41.309154 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:41.309159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:41.309378 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:41.309486 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:41.309492 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:41.310539 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:41.310543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:41.311171 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:41.311180 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:41.311717 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:41.311720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:41.311832 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:41.312475 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:41.312488 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:41.314723 (+ 2235us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":119,"threads_started":1}
W20260808 14:39:41.314980 29579 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:41.308285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:60615 (local address 127.19.144.126:36269)
0808 14:39:41.308444 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:41.308449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:41.308699 (+ 250us) server_negotiation.cc:407] Connection header received
0808 14:39:41.308931 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:41.308937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:41.309016 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:41.309105 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:41.309669 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:41.310374 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:41.311331 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:41.311588 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:41.311722 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:41.314866 (+ 3144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:60615: BlockingRecv error: recv got EOF from 127.19.144.66:60615 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 14:39:41 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:41 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.67@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:41.519407 29581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:41.513879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.67:52075)
0808 14:39:41.514155 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:41.514177 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:41.514276 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:41.514512 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:41.514515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:41.514701 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:41.514803 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:41.514809 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:41.515562 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:41.515565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:41.516155 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:41.516164 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:41.516600 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:41.516602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:41.516664 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:41.517162 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:41.517177 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:41.519280 (+ 2103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":110,"threads_started":1}
W20260808 14:39:41.519596 29579 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:41.513953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:52075 (local address 127.19.144.126:36269)
0808 14:39:41.514117 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:41.514121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:41.514246 (+ 125us) server_negotiation.cc:407] Connection header received
0808 14:39:41.514342 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:41.514345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:41.514394 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:41.514464 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:41.514907 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:41.515453 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:41.516270 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:41.516493 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:41.516580 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:41.519448 (+ 2868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.67:52075: BlockingRecv error: recv got EOF from 127.19.144.67:52075 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 14:39:42 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:42 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.65@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:42.133934 29582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:42.127406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.65:46881)
0808 14:39:42.127704 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:42.127718 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:42.127853 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:42.128224 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:42.128227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:42.128528 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:42.128644 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:42.128650 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:42.129750 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:42.129753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:42.130328 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:42.130337 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:42.130766 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:42.130768 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:42.130833 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:42.131349 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:42.131361 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:42.133789 (+ 2428us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":100,"threads_started":1}
W20260808 14:39:42.134083 29583 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:42.127577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:46881 (local address 127.19.144.126:36269)
0808 14:39:42.127854 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:42.127858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:42.127875 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:39:42.128003 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:42.128006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:42.128061 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:42.128178 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:42.128809 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:42.129563 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:42.130444 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:42.130665 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:42.130746 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:42.133918 (+ 3172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.65:46881: BlockingRecv error: recv got EOF from 127.19.144.65:46881 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":79,"threads_started":1}
Aug 08 14:39:42 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 14:39:42 dist-test-slave-p3t5 krb5kdc[28480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.144.66@KRBTEST.COM for oryx/127.19.144.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 14:39:42.322463 29584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:42.315992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:36269 (local address 127.19.144.66:33759)
0808 14:39:42.316359 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:42.316394 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:42.316549 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:42.316848 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:42.316851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:42.317083 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:42.317182 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:42.317187 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:42.318120 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:42.318123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:42.318843 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:42.318854 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:42.319336 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:42.319340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:42.319450 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:42.320010 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:42.320021 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:42.322261 (+ 2240us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.144.126:36269: Server oryx/127.19.144.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":93,"threads_started":1}
W20260808 14:39:42.322728 29583 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:42.316154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:33759 (local address 127.19.144.126:36269)
0808 14:39:42.316359 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:42.316365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:42.316495 (+ 130us) server_negotiation.cc:407] Connection header received
0808 14:39:42.316667 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:42.316670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:42.316724 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:42.316820 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:42.317313 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:42.317943 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:42.318972 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:42.319206 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:42.319320 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:42.322502 (+ 3182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.144.66:33759: BlockingRecv error: recv got EOF from 127.19.144.66:33759 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 14:39:42.394093 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 29088
I20260808 14:39:42.403853 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 29243
I20260808 14:39:42.411332 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 29384
I20260808 14:39:42.418788 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 29016
2026-08-08T14:39:42Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18185 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T14:39:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:42Z Disabled control of system clock
I20260808 14:39:42.454731 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:37987
--webserver_interface=127.19.144.126
--webserver_port=0
--builtin_ntp_servers=127.19.144.84:35519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:37987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 14:39:42.579442 29588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:42.579810 29588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:42.579912 29588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:42.584276 29588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:42.584394 29588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:42.584424 29588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:42.584467 29588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:42.584522 29588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:39:42.589999 29588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:35519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:37987
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:37987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:42.591352 29588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:42.592921 29588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:42.602617 29596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:42.602625 29594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:42.602690 29588 server_base.cc:1034] running on GCE node
W20260808 14:39:42.602623 29593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:42.603502 29588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:42.604861 29588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:42.606158 29588 hybrid_clock.cc:648] HybridClock initialized: now 1786199982606146 us; error 37 us; skew 500 ppm
I20260808 14:39:42.608408 29588 webserver.cc:460] Webserver started at http://127.19.144.126:32977/ using document root <none> and password file <none>
I20260808 14:39:42.609088 29588 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:42.609193 29588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:42.609455 29588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:42.611809 29588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "fb4f2e867271475f94b9a27048f8ed41"
format_stamp: "Formatted at 2026-08-08 14:39:42 on dist-test-slave-p3t5"
server_key: "df65793e6e13a5597bf09769004ff955"
server_key_iv: "84f74c6ab0b3e5322f673c4db7515e6e"
server_key_version: "encryptionkey@0"
I20260808 14:39:42.612458 29588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "fb4f2e867271475f94b9a27048f8ed41"
format_stamp: "Formatted at 2026-08-08 14:39:42 on dist-test-slave-p3t5"
server_key: "df65793e6e13a5597bf09769004ff955"
server_key_iv: "84f74c6ab0b3e5322f673c4db7515e6e"
server_key_version: "encryptionkey@0"
I20260808 14:39:42.616662 29588 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:39:42.619555 29602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:42.620995 29588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:39:42.621187 29588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "fb4f2e867271475f94b9a27048f8ed41"
format_stamp: "Formatted at 2026-08-08 14:39:42 on dist-test-slave-p3t5"
server_key: "df65793e6e13a5597bf09769004ff955"
server_key_iv: "84f74c6ab0b3e5322f673c4db7515e6e"
server_key_version: "encryptionkey@0"
I20260808 14:39:42.621335 29588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:42.641499 29588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:42.642556 29588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:42.642796 29588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:42.651331 29588 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:37987
I20260808 14:39:42.651336 29656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:37987 every 8 connection(s)
I20260808 14:39:42.652418 29588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:42.655966 29657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:42.662045 29657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41: Bootstrap starting.
I20260808 14:39:42.662146 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 29588
I20260808 14:39:42.662271 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:39:42.662583 20033 external_mini_cluster.cc:1428] Setting key f54f531444398f7351dabd432a65d37f
I20260808 14:39:42.664944 29657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:42.665863 29657 log.cc:824] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:42.668501 29657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41: No bootstrap required, opened a new log
I20260808 14:39:42.673782 29660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:42.664176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39322 (local address 127.19.144.126:37987)
0808 14:39:42.664874 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:42.664893 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:42.664931 (+ 38us) server_negotiation.cc:407] Connection header received
0808 14:39:42.665750 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:42.665772 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:42.666263 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:42.666576 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:42.667060 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:42.668502 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:42.669552 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:42.670314 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:42.670478 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:42.671389 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:42.671398 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:42.671416 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:42.671970 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:42.672057 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:42.672084 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:42.672203 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:42.672349 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 14:39:42.672710 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":162,"threads_started":1}
I20260808 14:39:42.673568 29657 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4f2e867271475f94b9a27048f8ed41" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37987 } }
I20260808 14:39:42.673902 29657 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:42.673961 29657 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb4f2e867271475f94b9a27048f8ed41, State: Initialized, Role: FOLLOWER
I20260808 14:39:42.674460 29657 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4f2e867271475f94b9a27048f8ed41" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37987 } }
I20260808 14:39:42.674626 29657 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:42.674752 29657 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:42.674865 29657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:42.675894 29657 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4f2e867271475f94b9a27048f8ed41" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37987 } }
I20260808 14:39:42.676281 29657 leader_election.cc:302] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb4f2e867271475f94b9a27048f8ed41; no voters:
I20260808 14:39:42.676632 29657 leader_election.cc:288] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:39:42.676826 29663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:42.677105 29663 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [term 1 LEADER]: Becoming Leader. State: Replica: fb4f2e867271475f94b9a27048f8ed41, State: Running, Role: LEADER
I20260808 14:39:42.677425 29663 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4f2e867271475f94b9a27048f8ed41" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37987 } }
I20260808 14:39:42.678035 29657 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:42.679188 29665 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb4f2e867271475f94b9a27048f8ed41. Latest consensus state: current_term: 1 leader_uuid: "fb4f2e867271475f94b9a27048f8ed41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4f2e867271475f94b9a27048f8ed41" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37987 } } }
I20260808 14:39:42.679330 29665 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:42.679669 29664 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb4f2e867271475f94b9a27048f8ed41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4f2e867271475f94b9a27048f8ed41" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37987 } } }
I20260808 14:39:42.679818 29664 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41 [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:42.679737 29672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:42.683344 29672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:42.689759 29672 catalog_manager.cc:1347] Generated new cluster ID: 9c7d43b3979c4a82a1b5bdfdf5f0dca9
I20260808 14:39:42.689852 29672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:42.701848 29672 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:39:42.703008 29672 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:42.709697 29672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb4f2e867271475f94b9a27048f8ed41: Generated new TSK 0
I20260808 14:39:42.710672 29672 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:39:42.720157 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:37987
--builtin_ntp_servers=127.19.144.84:35519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 14:39:42.858220 29682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:42.858580 29682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:42.858620 29682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:42.862874 29682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:42.862962 29682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:42.863019 29682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:39:42.868013 29682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:35519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:37987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:42.869305 29682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:42.870898 29682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:42.881495 29687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:42.881544 29690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:42.881515 29688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:42.881762 29682 server_base.cc:1034] running on GCE node
I20260808 14:39:42.882313 29682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:42.883037 29682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:42.884271 29682 hybrid_clock.cc:648] HybridClock initialized: now 1786199982884256 us; error 35 us; skew 500 ppm
I20260808 14:39:42.886392 29682 webserver.cc:460] Webserver started at http://127.19.144.65:36071/ using document root <none> and password file <none>
I20260808 14:39:42.887086 29682 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:42.887188 29682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:42.887454 29682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:42.889802 29682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "c1d73fcf1d154edebf2f2b837df26330"
format_stamp: "Formatted at 2026-08-08 14:39:42 on dist-test-slave-p3t5"
server_key: "bed87fd18d013bccb246d5b651767eb6"
server_key_iv: "85fd99b1047ea4c17ce1b0cc2aac72e0"
server_key_version: "encryptionkey@0"
I20260808 14:39:42.890439 29682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "c1d73fcf1d154edebf2f2b837df26330"
format_stamp: "Formatted at 2026-08-08 14:39:42 on dist-test-slave-p3t5"
server_key: "bed87fd18d013bccb246d5b651767eb6"
server_key_iv: "85fd99b1047ea4c17ce1b0cc2aac72e0"
server_key_version: "encryptionkey@0"
I20260808 14:39:42.894672 29682 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:39:42.897462 29696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:42.898856 29682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:42.899044 29682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "c1d73fcf1d154edebf2f2b837df26330"
format_stamp: "Formatted at 2026-08-08 14:39:42 on dist-test-slave-p3t5"
server_key: "bed87fd18d013bccb246d5b651767eb6"
server_key_iv: "85fd99b1047ea4c17ce1b0cc2aac72e0"
server_key_version: "encryptionkey@0"
I20260808 14:39:42.899180 29682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:42.924293 29682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:42.925115 29682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:42.925300 29682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:42.925983 29682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:42.927033 29682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:42.927083 29682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:42.927125 29682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:42.927140 29682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:42.936196 29682 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:42069
I20260808 14:39:42.936226 29812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:42069 every 8 connection(s)
I20260808 14:39:42.937171 29682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:39:42.939813 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 29682
I20260808 14:39:42.939963 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:39:42.940264 20033 external_mini_cluster.cc:1428] Setting key 94f255fba72b11e6986cff9c7b5c549c
I20260808 14:39:42.942533 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:37987
--builtin_ntp_servers=127.19.144.84:35519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 14:39:42.948239 29660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:42.939134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:48775 (local address 127.19.144.126:37987)
0808 14:39:42.939277 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:42.939281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:42.940421 (+ 1140us) server_negotiation.cc:407] Connection header received
0808 14:39:42.942099 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:42.942103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:42.942155 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:42.942249 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:42.943283 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:42.944165 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:42.945763 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:42.946166 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:42.946299 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:42.947547 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:42.947553 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:42.947559 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:42.947581 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:42.947610 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:42.947624 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:42.947709 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:42.947926 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 14:39:42.948010 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 14:39:42.949085 29815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:42.939480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:37987 (local address 127.19.144.65:48775)
0808 14:39:42.940255 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:42.940310 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:42.941873 (+ 1563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:42.942420 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:42.942427 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:42.942514 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:42.943127 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:42.943142 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:42.944326 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:42.944332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:42.945591 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:42.945603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:42.946350 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:42.946353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:42.946529 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:42.947243 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:39:42.947245 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:42.947290 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:39:42.947367 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:42.947727 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:42.947738 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:42.947747 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 14:39:42.947880 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 14:39:42.948149 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":154,"threads_started":1}
I20260808 14:39:42.950559 29813 heartbeater.cc:347] Connected to a master server at 127.19.144.126:37987
I20260808 14:39:42.950862 29813 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:42.951656 29813 heartbeater.cc:511] Master 127.19.144.126:37987 requested a full tablet report, sending...
I20260808 14:39:42.953501 29619 ts_manager.cc:194] Registered new tserver with Master: c1d73fcf1d154edebf2f2b837df26330 (127.19.144.65:42069)
I20260808 14:39:42.954766 29619 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.144.65:48775
W20260808 14:39:43.064363 29816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:43.064661 29816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:43.064699 29816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:43.068951 29816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:43.069028 29816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:43.069123 29816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:39:43.074162 29816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:35519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:37987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:43.075346 29816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:43.076870 29816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:43.087850 29824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:43.087805 29822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:43.087990 29816 server_base.cc:1034] running on GCE node
W20260808 14:39:43.088209 29821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:43.088687 29816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:43.089398 29816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:43.090571 29816 hybrid_clock.cc:648] HybridClock initialized: now 1786199983090555 us; error 73 us; skew 500 ppm
I20260808 14:39:43.092689 29816 webserver.cc:460] Webserver started at http://127.19.144.66:35429/ using document root <none> and password file <none>
I20260808 14:39:43.093336 29816 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:43.093405 29816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:43.093724 29816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:43.096000 29816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "d3d8231ba36b45bc9d6cde7df257698a"
format_stamp: "Formatted at 2026-08-08 14:39:43 on dist-test-slave-p3t5"
server_key: "5e98fb58e83b5a32e8024676797cd772"
server_key_iv: "7b38f990c94a3b8ac1c2467aacc6dfab"
server_key_version: "encryptionkey@0"
I20260808 14:39:43.096622 29816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "d3d8231ba36b45bc9d6cde7df257698a"
format_stamp: "Formatted at 2026-08-08 14:39:43 on dist-test-slave-p3t5"
server_key: "5e98fb58e83b5a32e8024676797cd772"
server_key_iv: "7b38f990c94a3b8ac1c2467aacc6dfab"
server_key_version: "encryptionkey@0"
I20260808 14:39:43.100961 29816 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:39:43.104209 29830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:43.105878 29816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:43.106052 29816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "d3d8231ba36b45bc9d6cde7df257698a"
format_stamp: "Formatted at 2026-08-08 14:39:43 on dist-test-slave-p3t5"
server_key: "5e98fb58e83b5a32e8024676797cd772"
server_key_iv: "7b38f990c94a3b8ac1c2467aacc6dfab"
server_key_version: "encryptionkey@0"
I20260808 14:39:43.106204 29816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:43.124457 29816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:43.125305 29816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:43.125520 29816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:43.126230 29816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:43.127264 29816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:43.127349 29816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:43.127425 29816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:43.127477 29816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:43.137121 29816 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:42051
I20260808 14:39:43.137140 29948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:42051 every 8 connection(s)
I20260808 14:39:43.138235 29816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:39:43.141494 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 29816
I20260808 14:39:43.141649 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:39:43.142042 20033 external_mini_cluster.cc:1428] Setting key 74b2d172c2117018c2286c5c5356fd58
I20260808 14:39:43.144219 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:37987
--builtin_ntp_servers=127.19.144.84:35519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 14:39:43.149435 29660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:43.140314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:38743 (local address 127.19.144.126:37987)
0808 14:39:43.140477 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:43.140482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:43.141755 (+ 1273us) server_negotiation.cc:407] Connection header received
0808 14:39:43.143429 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:43.143432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:43.143492 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:43.143559 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:43.144631 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:43.145479 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:43.147065 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:43.147476 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:43.147614 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:43.148760 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:43.148768 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:43.148774 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:43.148794 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:43.148823 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:43.148836 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:43.148920 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:43.149080 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 14:39:43.149167 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:39:43.150385 29951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:43.140818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:37987 (local address 127.19.144.66:38743)
0808 14:39:43.141532 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:43.141613 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:43.143218 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:43.143751 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:43.143759 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:43.143851 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:43.144477 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:43.144494 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:43.145681 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:43.145687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:43.146929 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:43.146941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:43.147641 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:43.147644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:43.147799 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:43.148470 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:39:43.148471 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:43.148515 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:39:43.148590 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:43.148927 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:43.148933 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:43.148938 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 14:39:43.149040 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 14:39:43.149307 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":125,"threads_started":1}
I20260808 14:39:43.151680 29949 heartbeater.cc:347] Connected to a master server at 127.19.144.126:37987
I20260808 14:39:43.151983 29949 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:43.152750 29949 heartbeater.cc:511] Master 127.19.144.126:37987 requested a full tablet report, sending...
I20260808 14:39:43.154065 29619 ts_manager.cc:194] Registered new tserver with Master: d3d8231ba36b45bc9d6cde7df257698a (127.19.144.66:42051)
I20260808 14:39:43.154692 29619 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.144.66:38743
W20260808 14:39:43.267321 29952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:43.267648 29952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:43.267689 29952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:43.271916 29952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:43.271986 29952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:43.272044 29952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:39:43.277055 29952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:35519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:37987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:43.278431 29952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:43.280162 29952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:43.291164 29957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:43.291147 29958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:43.291196 29952 server_base.cc:1034] running on GCE node
W20260808 14:39:43.291149 29961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:43.291895 29952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:43.292627 29952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:43.293833 29952 hybrid_clock.cc:648] HybridClock initialized: now 1786199983293798 us; error 47 us; skew 500 ppm
I20260808 14:39:43.296182 29952 webserver.cc:460] Webserver started at http://127.19.144.67:42383/ using document root <none> and password file <none>
I20260808 14:39:43.296960 29952 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:43.297067 29952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:43.297343 29952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:43.299851 29952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "9243a6be99314501b86180b89585ea9d"
format_stamp: "Formatted at 2026-08-08 14:39:43 on dist-test-slave-p3t5"
server_key: "1ed4b412df28232036af04b2bfeadffd"
server_key_iv: "067003e042e5ccce9c4623698e0cc6d3"
server_key_version: "encryptionkey@0"
I20260808 14:39:43.300519 29952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "9243a6be99314501b86180b89585ea9d"
format_stamp: "Formatted at 2026-08-08 14:39:43 on dist-test-slave-p3t5"
server_key: "1ed4b412df28232036af04b2bfeadffd"
server_key_iv: "067003e042e5ccce9c4623698e0cc6d3"
server_key_version: "encryptionkey@0"
I20260808 14:39:43.305233 29952 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:39:43.308274 29968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:43.309763 29952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:43.309937 29952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "9243a6be99314501b86180b89585ea9d"
format_stamp: "Formatted at 2026-08-08 14:39:43 on dist-test-slave-p3t5"
server_key: "1ed4b412df28232036af04b2bfeadffd"
server_key_iv: "067003e042e5ccce9c4623698e0cc6d3"
server_key_version: "encryptionkey@0"
I20260808 14:39:43.310081 29952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:43.326653 29952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:43.327495 29952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:43.327724 29952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:43.328410 29952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:43.329537 29952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:43.329612 29952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:43.329726 29952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:43.329782 29952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:43.339604 29952 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:40663
I20260808 14:39:43.339628 30081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:40663 every 8 connection(s)
I20260808 14:39:43.340679 29952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:39:43.343217 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 29952
I20260808 14:39:43.343314 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:39:43.343662 20033 external_mini_cluster.cc:1428] Setting key 34fe9e38f502090a1c852e9895c0f5d7
I20260808 14:39:43.351311 29660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:43.342840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:57725 (local address 127.19.144.126:37987)
0808 14:39:43.343050 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:43.343056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:43.344456 (+ 1400us) server_negotiation.cc:407] Connection header received
0808 14:39:43.346114 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:43.346118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:43.346177 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:43.346257 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:43.347308 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:43.347888 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:43.349077 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:43.349300 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:43.349383 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:43.350701 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:43.350705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:43.350707 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:43.350722 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:43.350747 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:43.350756 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:43.350830 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:43.351074 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 14:39:43.351159 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 14:39:43.352190 30084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:43.343393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:37987 (local address 127.19.144.67:57725)
0808 14:39:43.344285 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:43.344346 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:43.345827 (+ 1481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:43.346456 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:43.346464 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:43.346560 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:43.347156 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:43.347171 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:43.348013 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:43.348015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:43.348966 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:43.348976 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:43.349454 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:43.349457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:43.349608 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:43.350404 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:39:43.350405 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:43.350451 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:39:43.350525 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:43.350848 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:43.350855 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:43.350881 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 14:39:43.351006 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 14:39:43.351266 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":172,"threads_started":1}
I20260808 14:39:43.353564 30082 heartbeater.cc:347] Connected to a master server at 127.19.144.126:37987
I20260808 14:39:43.353853 30082 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:43.354422 30082 heartbeater.cc:511] Master 127.19.144.126:37987 requested a full tablet report, sending...
I20260808 14:39:43.355611 29619 ts_manager.cc:194] Registered new tserver with Master: 9243a6be99314501b86180b89585ea9d (127.19.144.67:40663)
I20260808 14:39:43.356187 29619 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.144.67:57725
I20260808 14:39:43.358150 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 14:39:43.361339 29660 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:43.360298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39336 (local address 127.19.144.126:37987)
0808 14:39:43.360456 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:43.360461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:43.360639 (+ 178us) server_negotiation.cc:407] Connection header received
0808 14:39:43.360751 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:43.360753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:43.360796 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:43.360865 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:43.361163 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39336: BlockingRecv error: recv got EOF from 127.0.0.1:39336 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:39:43.361536 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 29682
I20260808 14:39:43.368675 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 29816
I20260808 14:39:43.376847 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 29952
I20260808 14:39:43.385244 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 29588
2026-08-08T14:39:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (964 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T14:39:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:43Z Disabled control of system clock
I20260808 14:39:43.419143 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:40179
--webserver_interface=127.19.144.126
--webserver_port=0
--builtin_ntp_servers=127.19.144.84:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:40179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 14:39:43.540501 30093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:43.540813 30093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:43.540854 30093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:43.544991 30093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:43.545071 30093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:43.545086 30093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:43.545104 30093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:43.545120 30093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:39:43.550233 30093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:40179
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:40179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:43.551416 30093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:43.552814 30093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:43.561874 30098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:43.561992 30101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:43.562081 30093 server_base.cc:1034] running on GCE node
W20260808 14:39:43.561982 30099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:43.562829 30093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:43.563947 30093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:43.565117 30093 hybrid_clock.cc:648] HybridClock initialized: now 1786199983565093 us; error 39 us; skew 500 ppm
I20260808 14:39:43.567255 30093 webserver.cc:460] Webserver started at http://127.19.144.126:41015/ using document root <none> and password file <none>
I20260808 14:39:43.567907 30093 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:43.567973 30093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:43.568151 30093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:43.570550 30093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "c2ec48796e4e431ba39dd81855e82daa"
format_stamp: "Formatted at 2026-08-08 14:39:43 on dist-test-slave-p3t5"
server_key: "bb9016dc75933d19b57ce7283c3e56f8"
server_key_iv: "dc703349d217034c18fc98765f7d228f"
server_key_version: "encryptionkey@0"
I20260808 14:39:43.571113 30093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "c2ec48796e4e431ba39dd81855e82daa"
format_stamp: "Formatted at 2026-08-08 14:39:43 on dist-test-slave-p3t5"
server_key: "bb9016dc75933d19b57ce7283c3e56f8"
server_key_iv: "dc703349d217034c18fc98765f7d228f"
server_key_version: "encryptionkey@0"
I20260808 14:39:43.575022 30093 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:39:43.577486 30108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:43.578756 30093 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:39:43.578931 30093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "c2ec48796e4e431ba39dd81855e82daa"
format_stamp: "Formatted at 2026-08-08 14:39:43 on dist-test-slave-p3t5"
server_key: "bb9016dc75933d19b57ce7283c3e56f8"
server_key_iv: "dc703349d217034c18fc98765f7d228f"
server_key_version: "encryptionkey@0"
I20260808 14:39:43.579059 30093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:43.583803 30093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:43.584450 30093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:43.584643 30093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:43.591789 30093 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:40179
I20260808 14:39:43.591789 30163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:40179 every 8 connection(s)
I20260808 14:39:43.592808 30093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:43.596374 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 30093
I20260808 14:39:43.596498 30164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:43.596517 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:39:43.596833 20033 external_mini_cluster.cc:1428] Setting key 91ba3cf65fb917339f56cd0216147cd2
I20260808 14:39:43.603891 30167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:43.598404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55656 (local address 127.19.144.126:40179)
0808 14:39:43.599161 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:43.599190 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:43.599239 (+ 49us) server_negotiation.cc:407] Connection header received
0808 14:39:43.600223 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:43.600253 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:43.600778 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:43.601194 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:43.602203 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:43.602208 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:43.602225 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:43.602238 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:43.602312 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:43.602341 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:43.602514 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:43.602593 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 14:39:43.602850 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":143,"threads_started":1}
I20260808 14:39:43.604614 30164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa: Bootstrap starting.
I20260808 14:39:43.607198 30164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:43.607944 30164 log.cc:824] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:43.610095 30164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa: No bootstrap required, opened a new log
I20260808 14:39:43.613910 30164 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2ec48796e4e431ba39dd81855e82daa" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 40179 } }
I20260808 14:39:43.614153 30164 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:43.614188 30164 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2ec48796e4e431ba39dd81855e82daa, State: Initialized, Role: FOLLOWER
I20260808 14:39:43.614629 30164 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2ec48796e4e431ba39dd81855e82daa" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 40179 } }
I20260808 14:39:43.614759 30164 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:43.614812 30164 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:43.614902 30164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:43.615834 30164 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2ec48796e4e431ba39dd81855e82daa" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 40179 } }
I20260808 14:39:43.616165 30164 leader_election.cc:302] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2ec48796e4e431ba39dd81855e82daa; no voters:
I20260808 14:39:43.616466 30164 leader_election.cc:288] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:39:43.616612 30169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:43.616906 30169 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [term 1 LEADER]: Becoming Leader. State: Replica: c2ec48796e4e431ba39dd81855e82daa, State: Running, Role: LEADER
I20260808 14:39:43.617265 30169 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2ec48796e4e431ba39dd81855e82daa" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 40179 } }
I20260808 14:39:43.617870 30164 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:43.619259 30171 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2ec48796e4e431ba39dd81855e82daa. Latest consensus state: current_term: 1 leader_uuid: "c2ec48796e4e431ba39dd81855e82daa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2ec48796e4e431ba39dd81855e82daa" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 40179 } } }
I20260808 14:39:43.619418 30171 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:43.619805 30178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:43.619760 30170 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2ec48796e4e431ba39dd81855e82daa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2ec48796e4e431ba39dd81855e82daa" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 40179 } } }
I20260808 14:39:43.620143 30170 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:43.623068 30178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:43.629333 30178 catalog_manager.cc:1347] Generated new cluster ID: 5e4324cb4c72431f843083bc93aa502c
I20260808 14:39:43.629423 30178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:43.651049 30178 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:39:43.651470 30178 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:43.662120 30178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2ec48796e4e431ba39dd81855e82daa: Generated new TSK 0
I20260808 14:39:43.663097 30178 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:39:43.676075 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:40179
--builtin_ntp_servers=127.19.144.84:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 14:39:43.796653 30188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:43.796978 30188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:43.797022 30188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:43.801285 30188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:43.801366 30188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:43.801460 30188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:39:43.806589 30188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:40179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:43.807945 30188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:43.809520 30188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:43.820315 30194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:43.820312 30193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:43.820387 30196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:43.820747 30188 server_base.cc:1034] running on GCE node
I20260808 14:39:43.821290 30188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:43.822098 30188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:43.823303 30188 hybrid_clock.cc:648] HybridClock initialized: now 1786199983823282 us; error 32 us; skew 500 ppm
I20260808 14:39:43.825448 30188 webserver.cc:460] Webserver started at http://127.19.144.65:38525/ using document root <none> and password file <none>
I20260808 14:39:43.826175 30188 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:43.826282 30188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:43.826514 30188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:43.828759 30188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "bfdf611eeeed48e7ab8aece829263370"
format_stamp: "Formatted at 2026-08-08 14:39:43 on dist-test-slave-p3t5"
server_key: "c032a65fe0b7077de0f75fd0d0320be1"
server_key_iv: "33427074fe965141b0ae2f2d87200a7b"
server_key_version: "encryptionkey@0"
I20260808 14:39:43.829391 30188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "bfdf611eeeed48e7ab8aece829263370"
format_stamp: "Formatted at 2026-08-08 14:39:43 on dist-test-slave-p3t5"
server_key: "c032a65fe0b7077de0f75fd0d0320be1"
server_key_iv: "33427074fe965141b0ae2f2d87200a7b"
server_key_version: "encryptionkey@0"
I20260808 14:39:43.833443 30188 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 14:39:43.836452 30202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:43.837898 30188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:43.838059 30188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "bfdf611eeeed48e7ab8aece829263370"
format_stamp: "Formatted at 2026-08-08 14:39:43 on dist-test-slave-p3t5"
server_key: "c032a65fe0b7077de0f75fd0d0320be1"
server_key_iv: "33427074fe965141b0ae2f2d87200a7b"
server_key_version: "encryptionkey@0"
I20260808 14:39:43.838188 30188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:43.843060 30188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:43.843736 30188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:43.843948 30188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:43.844604 30188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:43.845686 30188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:43.845757 30188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:43.845834 30188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:43.845885 30188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:43.855154 30188 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:43047
I20260808 14:39:43.855173 30318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:43047 every 8 connection(s)
I20260808 14:39:43.856190 30188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:39:43.862733 30167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:43.858185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:41015 (local address 127.19.144.126:40179)
0808 14:39:43.858345 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:43.858349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:43.859535 (+ 1186us) server_negotiation.cc:407] Connection header received
0808 14:39:43.860941 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:43.860944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:43.860988 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:43.861069 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:43.862323 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:43.862325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:43.862328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:43.862339 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:43.862356 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:43.862364 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:43.862441 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:43.862577 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 14:39:43.862618 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:39:43.863691 30322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:43.858635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:40179 (local address 127.19.144.65:41015)
0808 14:39:43.859351 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:43.859399 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:43.860651 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:43.861256 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:43.861264 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:43.861362 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:43.862061 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:39:43.862062 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:43.862108 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:39:43.862184 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:43.862442 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:43.862450 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:43.862454 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 14:39:43.862596 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 14:39:43.862828 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":114,"threads_started":1}
I20260808 14:39:43.864500 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 30188
I20260808 14:39:43.864647 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:39:43.865015 20033 external_mini_cluster.cc:1428] Setting key ea188c75ca9d2d57cadd75fafa1821cb
I20260808 14:39:43.865242 30319 heartbeater.cc:347] Connected to a master server at 127.19.144.126:40179
I20260808 14:39:43.865548 30319 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:43.866113 30319 heartbeater.cc:511] Master 127.19.144.126:40179 requested a full tablet report, sending...
I20260808 14:39:43.867384 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:40179
--builtin_ntp_servers=127.19.144.84:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 14:39:43.868054 30127 ts_manager.cc:194] Registered new tserver with Master: bfdf611eeeed48e7ab8aece829263370 (127.19.144.65:43047)
W20260808 14:39:43.990269 30323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:43.990583 30323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:43.990626 30323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:43.994870 30323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:43.994941 30323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:43.995005 30323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:39:44.000007 30323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:40179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:44.001206 30323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:44.002722 30323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:44.013083 30331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:44.013062 30329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:44.013072 30328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:44.013266 30323 server_base.cc:1034] running on GCE node
I20260808 14:39:44.013929 30323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:44.014654 30323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:44.015853 30323 hybrid_clock.cc:648] HybridClock initialized: now 1786199984015824 us; error 56 us; skew 500 ppm
I20260808 14:39:44.018038 30323 webserver.cc:460] Webserver started at http://127.19.144.66:36999/ using document root <none> and password file <none>
I20260808 14:39:44.018687 30323 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:44.018800 30323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:44.019068 30323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:44.021314 30323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "61d454d921574bb9b2cfb551fd92adb0"
format_stamp: "Formatted at 2026-08-08 14:39:44 on dist-test-slave-p3t5"
server_key: "62ee6044c53eaa964e932c093115f801"
server_key_iv: "789b3cefa76767993064e7b83957268f"
server_key_version: "encryptionkey@0"
I20260808 14:39:44.021994 30323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "61d454d921574bb9b2cfb551fd92adb0"
format_stamp: "Formatted at 2026-08-08 14:39:44 on dist-test-slave-p3t5"
server_key: "62ee6044c53eaa964e932c093115f801"
server_key_iv: "789b3cefa76767993064e7b83957268f"
server_key_version: "encryptionkey@0"
I20260808 14:39:44.026095 30323 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:39:44.028883 30337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:44.030285 30323 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:44.030452 30323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "61d454d921574bb9b2cfb551fd92adb0"
format_stamp: "Formatted at 2026-08-08 14:39:44 on dist-test-slave-p3t5"
server_key: "62ee6044c53eaa964e932c093115f801"
server_key_iv: "789b3cefa76767993064e7b83957268f"
server_key_version: "encryptionkey@0"
I20260808 14:39:44.030582 30323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:44.035553 30323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:44.036216 30323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:44.036422 30323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:44.037010 30323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:44.038067 30323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:44.038141 30323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:44.038213 30323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:44.038266 30323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:44.047560 30323 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:37357
I20260808 14:39:44.047595 30454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:37357 every 8 connection(s)
I20260808 14:39:44.048590 30323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:39:44.055436 30167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:44.050763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:60877 (local address 127.19.144.126:40179)
0808 14:39:44.050917 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:44.050921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:44.052125 (+ 1204us) server_negotiation.cc:407] Connection header received
0808 14:39:44.053541 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:44.053544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:44.053593 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:44.053706 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:44.054873 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:44.054877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:44.054880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:44.054895 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:44.054923 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:44.054938 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:44.055045 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:44.055205 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 14:39:44.055259 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:39:44.055971 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 30323
I20260808 14:39:44.056125 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:39:44.056492 20033 external_mini_cluster.cc:1428] Setting key 48c44a6eef1480bc64b906231b3fd22b
I20260808 14:39:44.056687 30457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:44.051210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:40179 (local address 127.19.144.66:60877)
0808 14:39:44.051949 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:44.051996 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:44.053292 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:44.053905 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:44.053913 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:44.054011 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:44.054588 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:39:44.054590 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:44.054634 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:39:44.054708 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:44.055035 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:44.055044 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:44.055051 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 14:39:44.055253 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 14:39:44.055531 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":136,"threads_started":1}
I20260808 14:39:44.058187 30455 heartbeater.cc:347] Connected to a master server at 127.19.144.126:40179
I20260808 14:39:44.058488 30455 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:44.059016 30455 heartbeater.cc:511] Master 127.19.144.126:40179 requested a full tablet report, sending...
I20260808 14:39:44.059087 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:40179
--builtin_ntp_servers=127.19.144.84:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 14:39:44.060279 30127 ts_manager.cc:194] Registered new tserver with Master: 61d454d921574bb9b2cfb551fd92adb0 (127.19.144.66:37357)
W20260808 14:39:44.182674 30458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:44.183003 30458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:44.183090 30458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:44.187314 30458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:44.187414 30458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:44.187513 30458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:39:44.192541 30458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:40179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:44.193835 30458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:44.195393 30458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:44.205925 30466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:44.205948 30464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:44.205910 30463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:44.206167 30458 server_base.cc:1034] running on GCE node
I20260808 14:39:44.206847 30458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:44.207633 30458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:44.208846 30458 hybrid_clock.cc:648] HybridClock initialized: now 1786199984208812 us; error 43 us; skew 500 ppm
I20260808 14:39:44.211007 30458 webserver.cc:460] Webserver started at http://127.19.144.67:33977/ using document root <none> and password file <none>
I20260808 14:39:44.211635 30458 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:44.211704 30458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:44.211939 30458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:44.214182 30458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "3cea1fece9484a179576bba70a9f2752"
format_stamp: "Formatted at 2026-08-08 14:39:44 on dist-test-slave-p3t5"
server_key: "738fed40f1645abdf1d141edb0bf21ab"
server_key_iv: "b1bcbe37dfe2547e61d313872a730a11"
server_key_version: "encryptionkey@0"
I20260808 14:39:44.214788 30458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "3cea1fece9484a179576bba70a9f2752"
format_stamp: "Formatted at 2026-08-08 14:39:44 on dist-test-slave-p3t5"
server_key: "738fed40f1645abdf1d141edb0bf21ab"
server_key_iv: "b1bcbe37dfe2547e61d313872a730a11"
server_key_version: "encryptionkey@0"
I20260808 14:39:44.218935 30458 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 14:39:44.221740 30472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:44.223058 30458 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:39:44.223160 30458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "3cea1fece9484a179576bba70a9f2752"
format_stamp: "Formatted at 2026-08-08 14:39:44 on dist-test-slave-p3t5"
server_key: "738fed40f1645abdf1d141edb0bf21ab"
server_key_iv: "b1bcbe37dfe2547e61d313872a730a11"
server_key_version: "encryptionkey@0"
I20260808 14:39:44.223258 30458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:44.228032 30458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:44.228634 30458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:44.228787 30458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:44.229297 30458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:44.230281 30458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:44.230329 30458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:44.230368 30458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:44.230389 30458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:44.239926 30458 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:37347
I20260808 14:39:44.239961 30590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:37347 every 8 connection(s)
I20260808 14:39:44.240947 30458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:39:44.247164 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 30458
I20260808 14:39:44.247409 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:39:44.247743 20033 external_mini_cluster.cc:1428] Setting key 59a5c76adb4e7097dbfb6bc79a950b81
I20260808 14:39:44.247860 30167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:44.243075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:59201 (local address 127.19.144.126:40179)
0808 14:39:44.243215 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:44.243219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:44.244435 (+ 1216us) server_negotiation.cc:407] Connection header received
0808 14:39:44.245898 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:44.245902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:44.245952 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:44.246063 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:44.247297 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:44.247301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:44.247306 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:39:44.247325 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:44.247349 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:39:44.247358 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:44.247507 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:44.247664 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 14:39:44.247717 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:39:44.249001 30593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:44.243555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:40179 (local address 127.19.144.67:59201)
0808 14:39:44.244262 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:44.244305 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:44.245594 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:44.246254 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:44.246261 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:44.246361 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:44.247004 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:39:44.247005 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:44.247049 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:39:44.247126 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:44.247489 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:44.247499 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:44.247505 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 14:39:44.247709 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 14:39:44.247985 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":124,"threads_started":1}
I20260808 14:39:44.250278 30591 heartbeater.cc:347] Connected to a master server at 127.19.144.126:40179
I20260808 14:39:44.250547 30591 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:44.251009 30591 heartbeater.cc:511] Master 127.19.144.126:40179 requested a full tablet report, sending...
I20260808 14:39:44.252090 30127 ts_manager.cc:194] Registered new tserver with Master: 3cea1fece9484a179576bba70a9f2752 (127.19.144.67:37347)
I20260808 14:39:44.262246 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 14:39:44.265297 30167 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:39:44.264335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55670 (local address 127.19.144.126:40179)
0808 14:39:44.264496 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:44.264501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:44.264706 (+ 205us) server_negotiation.cc:407] Connection header received
0808 14:39:44.264792 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:44.264794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:44.264831 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:44.264902 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:44.265139 (+ 237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55670: BlockingRecv error: recv got EOF from 127.0.0.1:55670 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 14:39:44.265385 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 30188
I20260808 14:39:44.274031 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 30323
I20260808 14:39:44.280931 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 30458
I20260808 14:39:44.288062 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 30093
2026-08-08T14:39:44Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (902 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:39:44 dist-test-slave-p3t5 krb5kdc[30601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:39:44 dist-test-slave-p3t5 krb5kdc[30601](info): set up 2 sockets
Aug 08 14:39:44 dist-test-slave-p3t5 krb5kdc[30601](info): commencing operation
krb5kdc: starting...
W20260808 14:39:45.400906 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:39:45 dist-test-slave-p3t5 krb5kdc[30601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:39:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:39:45.541471 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:45751
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:45751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:45.670523 30614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:45.670842 30614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:45.670892 30614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:45.675207 30614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:39:45.675277 30614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:45.675298 30614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:45.675315 30614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:39:45.675355 30614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:39:45.681030 30614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:45751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:45751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:45.682268 30614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:45.683365 30614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:45.692214 30620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:45.692222 30619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:45.692217 30622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:45.692523 30614 server_base.cc:1034] running on GCE node
I20260808 14:39:45.693179 30614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:45.694550 30614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:45.695751 30614 hybrid_clock.cc:648] HybridClock initialized: now 1786199985695732 us; error 33 us; skew 500 ppm
Aug 08 14:39:45 dist-test-slave-p3t5 krb5kdc[30601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199985, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:45.698765 30614 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:39:45.699826 30614 webserver.cc:460] Webserver started at http://127.19.144.126:44227/ using document root <none> and password file <none>
I20260808 14:39:45.700456 30614 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:45.700557 30614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:45.700807 30614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:45.703058 30614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "3c7cea28395b4160b8cfb3d39fd1855d"
format_stamp: "Formatted at 2026-08-08 14:39:45 on dist-test-slave-p3t5"
server_key: "395afccc8a0e9efd56f3c77c212faea9"
server_key_iv: "04aac64d63a0931e445d17af99fe07d3"
server_key_version: "encryptionkey@0"
I20260808 14:39:45.703696 30614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "3c7cea28395b4160b8cfb3d39fd1855d"
format_stamp: "Formatted at 2026-08-08 14:39:45 on dist-test-slave-p3t5"
server_key: "395afccc8a0e9efd56f3c77c212faea9"
server_key_iv: "04aac64d63a0931e445d17af99fe07d3"
server_key_version: "encryptionkey@0"
I20260808 14:39:45.707752 30614 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:39:45.710387 30630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:45.711725 30614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:45.711887 30614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "3c7cea28395b4160b8cfb3d39fd1855d"
format_stamp: "Formatted at 2026-08-08 14:39:45 on dist-test-slave-p3t5"
server_key: "395afccc8a0e9efd56f3c77c212faea9"
server_key_iv: "04aac64d63a0931e445d17af99fe07d3"
server_key_version: "encryptionkey@0"
I20260808 14:39:45.712033 30614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:45.728305 30614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:45.732007 30614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:45.732247 30614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:45.740023 30614 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:45751
I20260808 14:39:45.740301 30684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:45751 every 8 connection(s)
I20260808 14:39:45.741158 30614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:39:45.744241 30685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:45.750355 30685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d: Bootstrap starting.
I20260808 14:39:45.750499 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 30614
I20260808 14:39:45.750663 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:39:45.750957 20033 external_mini_cluster.cc:1428] Setting key 1370d6e6a024b4d77cd9ed560b058483
I20260808 14:39:45.753212 30685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:45.754098 30685 log.cc:824] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:45.756480 30685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d: No bootstrap required, opened a new log
Aug 08 14:39:45 dist-test-slave-p3t5 krb5kdc[30601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:45.760723 30685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7cea28395b4160b8cfb3d39fd1855d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45751 } }
I20260808 14:39:45.761008 30685 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:45.761092 30685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c7cea28395b4160b8cfb3d39fd1855d, State: Initialized, Role: FOLLOWER
I20260808 14:39:45.761667 30685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7cea28395b4160b8cfb3d39fd1855d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45751 } }
I20260808 14:39:45.761842 30685 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:39:45.761915 30685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:39:45.762024 30685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:45.762998 30685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7cea28395b4160b8cfb3d39fd1855d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45751 } }
I20260808 14:39:45.763384 30685 leader_election.cc:302] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c7cea28395b4160b8cfb3d39fd1855d; no voters:
I20260808 14:39:45.763774 30685 leader_election.cc:288] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:39:45.763857 30690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:45.764073 30690 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [term 1 LEADER]: Becoming Leader. State: Replica: 3c7cea28395b4160b8cfb3d39fd1855d, State: Running, Role: LEADER
I20260808 14:39:45.764401 30690 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7cea28395b4160b8cfb3d39fd1855d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45751 } }
I20260808 14:39:45.765235 30685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:39:45.766271 30692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c7cea28395b4160b8cfb3d39fd1855d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7cea28395b4160b8cfb3d39fd1855d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45751 } } }
I20260808 14:39:45.766656 30692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [sys.catalog]: This master's current role is: LEADER
I20260808 14:39:45.766881 30688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:45.752483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59438 (local address 127.19.144.126:45751)
0808 14:39:45.752932 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:45.752948 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:45.752985 (+ 37us) server_negotiation.cc:407] Connection header received
0808 14:39:45.753798 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:45.753819 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:45.754180 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:45.754588 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:45.755458 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:45.756499 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:45.757418 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:45.757978 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:45.758155 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:45.761053 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:45.761060 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:45.761079 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:45.761114 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:45.763156 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:45.763706 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:45.763712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:45.763719 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:45.763883 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:45.764291 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:45.764295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:45.764297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:45.764803 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:45.765016 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:45.765317 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 14:39:45.765693 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":150,"threads_started":1}
I20260808 14:39:45.766561 30691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c7cea28395b4160b8cfb3d39fd1855d. Latest consensus state: current_term: 1 leader_uuid: "3c7cea28395b4160b8cfb3d39fd1855d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7cea28395b4160b8cfb3d39fd1855d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 45751 } } }
I20260808 14:39:45.768522 30691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d [sys.catalog]: This master's current role is: LEADER
W20260808 14:39:45.770618 30705 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:39:45.770702 30705 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:39:45.770790 30699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:39:45.771562 30699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:39:45.777863 30699 catalog_manager.cc:1347] Generated new cluster ID: 691138a1c5004009a8d5b3ad74cdc923
I20260808 14:39:45.777961 30699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:39:45.796559 30699 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:39:45.797756 30699 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:39:45.802804 30699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d: Generated new TSK 0
I20260808 14:39:45.803575 30699 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:39:45.843396 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:45751
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:45.965514 30711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:45.965862 30711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:45.965960 30711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:45.970465 30711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:45.970542 30711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:45.970638 30711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:39:45.976171 30711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:45751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:45.977439 30711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:45.978631 30711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:45.989427 30717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:45.989427 30720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:45.989537 30711 server_base.cc:1034] running on GCE node
W20260808 14:39:45.989420 30718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:45.990247 30711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:45.990986 30711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:45.992204 30711 hybrid_clock.cc:648] HybridClock initialized: now 1786199985992184 us; error 38 us; skew 500 ppm
Aug 08 14:39:45 dist-test-slave-p3t5 krb5kdc[30601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199985, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:45.995384 30711 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:39:45.996395 30711 webserver.cc:460] Webserver started at http://127.19.144.65:40595/ using document root <none> and password file <none>
I20260808 14:39:45.996978 30711 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:45.997042 30711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:45.997222 30711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:45.999503 30711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "4306ba6b42c845f4b1ff6f33ada8bf9f"
format_stamp: "Formatted at 2026-08-08 14:39:45 on dist-test-slave-p3t5"
server_key: "fee6e7c0daf8a782140cf552c13e949b"
server_key_iv: "afb4f9ae6eefc8164c1d09e562cd694f"
server_key_version: "encryptionkey@0"
I20260808 14:39:46.000092 30711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "4306ba6b42c845f4b1ff6f33ada8bf9f"
format_stamp: "Formatted at 2026-08-08 14:39:45 on dist-test-slave-p3t5"
server_key: "fee6e7c0daf8a782140cf552c13e949b"
server_key_iv: "afb4f9ae6eefc8164c1d09e562cd694f"
server_key_version: "encryptionkey@0"
I20260808 14:39:46.004192 30711 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 14:39:46.006990 30727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:46.008361 30711 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 14:39:46.008512 30711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "4306ba6b42c845f4b1ff6f33ada8bf9f"
format_stamp: "Formatted at 2026-08-08 14:39:45 on dist-test-slave-p3t5"
server_key: "fee6e7c0daf8a782140cf552c13e949b"
server_key_iv: "afb4f9ae6eefc8164c1d09e562cd694f"
server_key_version: "encryptionkey@0"
I20260808 14:39:46.008636 30711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:46.049201 30711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:46.053006 30711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:46.053246 30711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:46.053975 30711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:46.054919 30711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:46.054977 30711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:46.055066 30711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:46.055104 30711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:46.064734 30711 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:33333
I20260808 14:39:46.064759 30842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:33333 every 8 connection(s)
I20260808 14:39:46.065836 30711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:39:46.072907 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 30711
I20260808 14:39:46.073063 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:39:46.073385 20033 external_mini_cluster.cc:1428] Setting key d4cccdeaf0d28da83e26df78eb14beb1
Aug 08 14:39:46 dist-test-slave-p3t5 krb5kdc[30601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199985, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:46.082842 30688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.068265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:39055 (local address 127.19.144.126:45751)
0808 14:39:46.068427 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:46.068431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:46.069314 (+ 883us) server_negotiation.cc:407] Connection header received
0808 14:39:46.071014 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:46.071019 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:46.071089 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:46.071215 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:46.072718 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.073679 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.075371 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.075762 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.075899 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.078708 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:46.078714 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:46.078721 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:46.078747 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:46.080318 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:46.080917 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:46.080920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:46.080922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:46.081018 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:46.081451 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:46.081454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:46.081455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:46.081723 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:46.081813 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:46.082568 (+ 755us) server_negotiation.cc:299] Negotiation successful
0808 14:39:46.082689 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 14:39:46.084190 30845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.068606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45751 (local address 127.19.144.65:39055)
0808 14:39:46.069119 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:46.069169 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:46.070727 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:46.071437 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:46.071445 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:46.071880 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:46.072469 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:46.072484 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.073853 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.073858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:46.075219 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:46.075230 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.075933 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.075936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:46.076090 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.076764 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:46.076781 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:46.078521 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:46.080448 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:46.080455 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:46.080474 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:46.080798 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:46.081118 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:46.081124 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:46.081129 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:46.081337 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:46.081845 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:46.081856 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:46.082250 (+ 394us) client_negotiation.cc:769] Sending connection context
0808 14:39:46.082524 (+ 274us) client_negotiation.cc:240] Negotiation successful
0808 14:39:46.082970 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:39:46.085711 30843 heartbeater.cc:347] Connected to a master server at 127.19.144.126:45751
I20260808 14:39:46.085996 30843 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:46.086558 30843 heartbeater.cc:511] Master 127.19.144.126:45751 requested a full tablet report, sending...
I20260808 14:39:46.088351 30648 ts_manager.cc:194] Registered new tserver with Master: 4306ba6b42c845f4b1ff6f33ada8bf9f (127.19.144.65:33333)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
I20260808 14:39:46.089769 30648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:39055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:39:46.107065 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:45751
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:46.227797 30848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:46.228159 30848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:46.228250 30848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:46.232429 30848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:46.232558 30848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:46.232662 30848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:39:46.238261 30848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:45751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:46.239558 30848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:46.240736 30848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:46.251786 30856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:46.251847 30853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:46.251806 30854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:46.252063 30848 server_base.cc:1034] running on GCE node
I20260808 14:39:46.252570 30848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:46.253294 30848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:46.254515 30848 hybrid_clock.cc:648] HybridClock initialized: now 1786199986254476 us; error 50 us; skew 500 ppm
Aug 08 14:39:46 dist-test-slave-p3t5 krb5kdc[30601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199986, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:46.265758 30848 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:39:46.267230 30848 webserver.cc:460] Webserver started at http://127.19.144.66:37751/ using document root <none> and password file <none>
I20260808 14:39:46.267982 30848 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:46.268095 30848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:46.268349 30848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:46.270738 30848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "6bb9003f93d54d6e8502f3f4556f9144"
format_stamp: "Formatted at 2026-08-08 14:39:46 on dist-test-slave-p3t5"
server_key: "c71ec7b026b0215770f7683841605718"
server_key_iv: "d7d485494a97046c5fa61a8a2b0a00bb"
server_key_version: "encryptionkey@0"
I20260808 14:39:46.271387 30848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "6bb9003f93d54d6e8502f3f4556f9144"
format_stamp: "Formatted at 2026-08-08 14:39:46 on dist-test-slave-p3t5"
server_key: "c71ec7b026b0215770f7683841605718"
server_key_iv: "d7d485494a97046c5fa61a8a2b0a00bb"
server_key_version: "encryptionkey@0"
I20260808 14:39:46.275731 30848 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 14:39:46.278654 30863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:46.280100 30848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 14:39:46.280272 30848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "6bb9003f93d54d6e8502f3f4556f9144"
format_stamp: "Formatted at 2026-08-08 14:39:46 on dist-test-slave-p3t5"
server_key: "c71ec7b026b0215770f7683841605718"
server_key_iv: "d7d485494a97046c5fa61a8a2b0a00bb"
server_key_version: "encryptionkey@0"
I20260808 14:39:46.280418 30848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:46.319092 30848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:46.322697 30848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:46.322937 30848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:46.323601 30848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:46.324627 30848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:46.324702 30848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:46.324787 30848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:46.324832 30848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:46.334359 30848 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:43701
I20260808 14:39:46.334381 30977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:43701 every 8 connection(s)
I20260808 14:39:46.335354 30848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:39:46.336103 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 30848
I20260808 14:39:46.336249 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:39:46.336555 20033 external_mini_cluster.cc:1428] Setting key ed34ed9a0c9a0b7d5add42126b4a7d32
Aug 08 14:39:46 dist-test-slave-p3t5 krb5kdc[30601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199986, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
I20260808 14:39:46.350522 30688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.337414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:42957 (local address 127.19.144.126:45751)
0808 14:39:46.337568 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:46.337572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:46.338686 (+ 1114us) server_negotiation.cc:407] Connection header received
0808 14:39:46.340111 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:46.340115 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:46.340169 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:46.340304 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:46.341545 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.342137 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.343355 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.343596 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.343708 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.346483 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:46.346486 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:46.346488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:46.346515 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:46.348088 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:46.348700 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:46.348704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:46.348706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:46.348801 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:46.349161 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:46.349165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:46.349167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:46.349383 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:46.349506 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:46.350179 (+ 673us) server_negotiation.cc:299] Negotiation successful
0808 14:39:46.350349 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 14:39:46.351495 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.337992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45751 (local address 127.19.144.66:42957)
0808 14:39:46.338529 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:46.338575 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:46.339888 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:46.340485 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:46.340492 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:46.340884 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:46.341382 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:46.341396 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.342270 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.342276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:46.343214 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:46.343224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.343784 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.343790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:46.343975 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.344652 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:46.344670 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:46.346285 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:46.348225 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:46.348233 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:46.348252 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:46.348571 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:46.348897 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:46.348901 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:46.348903 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:46.349061 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:46.349518 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:46.349526 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:46.349928 (+ 402us) client_negotiation.cc:769] Sending connection context
0808 14:39:46.350177 (+ 249us) client_negotiation.cc:240] Negotiation successful
0808 14:39:46.350583 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":186,"threads_started":1}
I20260808 14:39:46.352890 30978 heartbeater.cc:347] Connected to a master server at 127.19.144.126:45751
I20260808 14:39:46.353184 30978 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:46.353811 30978 heartbeater.cc:511] Master 127.19.144.126:45751 requested a full tablet report, sending...
I20260808 14:39:46.355026 30648 ts_manager.cc:194] Registered new tserver with Master: 6bb9003f93d54d6e8502f3f4556f9144 (127.19.144.66:43701)
I20260808 14:39:46.355952 30648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:42957
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:39:46.369303 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:45751
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:39:46.488799 30983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:39:46.489099 30983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:39:46.489146 30983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:39:46.493234 30983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:39:46.493314 30983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:39:46.493373 30983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:39:46.498883 30983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:45751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:39:46.499989 30983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:39:46.501032 30983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:39:46.511153 30989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:46.511171 30988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:39:46.511160 30991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:39:46.511700 30983 server_base.cc:1034] running on GCE node
I20260808 14:39:46.512274 30983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:39:46.512971 30983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:39:46.514164 30983 hybrid_clock.cc:648] HybridClock initialized: now 1786199986514136 us; error 47 us; skew 500 ppm
Aug 08 14:39:46 dist-test-slave-p3t5 krb5kdc[30601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199986, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:39:46.517274 30983 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:39:46.518427 30983 webserver.cc:460] Webserver started at http://127.19.144.67:39815/ using document root <none> and password file <none>
I20260808 14:39:46.519061 30983 fs_manager.cc:359] Metadata directory not provided
I20260808 14:39:46.519160 30983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:39:46.519412 30983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:39:46.521685 30983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "2602bd22316f4b24a8af8a54816c58f5"
format_stamp: "Formatted at 2026-08-08 14:39:46 on dist-test-slave-p3t5"
server_key: "04dac307433206cf4e076b2d185d1412"
server_key_iv: "107f87343e565638f578d7d019d3b3da"
server_key_version: "encryptionkey@0"
I20260808 14:39:46.522337 30983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "2602bd22316f4b24a8af8a54816c58f5"
format_stamp: "Formatted at 2026-08-08 14:39:46 on dist-test-slave-p3t5"
server_key: "04dac307433206cf4e076b2d185d1412"
server_key_iv: "107f87343e565638f578d7d019d3b3da"
server_key_version: "encryptionkey@0"
I20260808 14:39:46.526940 30983 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:39:46.530311 30998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:46.531747 30983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:39:46.531876 30983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "2602bd22316f4b24a8af8a54816c58f5"
format_stamp: "Formatted at 2026-08-08 14:39:46 on dist-test-slave-p3t5"
server_key: "04dac307433206cf4e076b2d185d1412"
server_key_iv: "107f87343e565638f578d7d019d3b3da"
server_key_version: "encryptionkey@0"
I20260808 14:39:46.531984 30983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:39:46.553777 30983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:39:46.557720 30983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:39:46.557960 30983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:39:46.558686 30983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:39:46.560072 30983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:39:46.560154 30983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:46.560220 30983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:39:46.560256 30983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:39:46.572492 30983 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:34539
I20260808 14:39:46.572528 31114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:34539 every 8 connection(s)
I20260808 14:39:46.573521 30983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:39:46.578040 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 30983
I20260808 14:39:46.578207 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:39:46.578584 20033 external_mini_cluster.cc:1428] Setting key 2ef0e92d69182ce5642d410732773e38
Aug 08 14:39:46 dist-test-slave-p3t5 krb5kdc[30601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199986, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:39:46.589088 30688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.575599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:42063 (local address 127.19.144.126:45751)
0808 14:39:46.575763 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:46.575768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:46.576733 (+ 965us) server_negotiation.cc:407] Connection header received
0808 14:39:46.578319 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:46.578323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:46.578390 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:46.578496 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:46.580023 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.580841 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.582124 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.582345 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.582428 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.585157 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:46.585161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:46.585163 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:46.585191 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:46.586759 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:46.587362 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:46.587370 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:46.587371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:46.587459 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:46.587853 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:46.587856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:46.587858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:46.588072 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:46.588180 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:46.588760 (+ 580us) server_negotiation.cc:299] Negotiation successful
0808 14:39:46.588910 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:39:46.590085 31118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.576058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:45751 (local address 127.19.144.67:42063)
0808 14:39:46.576567 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:46.576616 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:46.578051 (+ 1435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:46.578666 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:46.578673 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:46.579208 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:39:46.579849 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:46.579866 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.580990 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.580993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:46.582015 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:46.582025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.582487 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.582489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:46.582648 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.583252 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:39:46.583270 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:39:46.584933 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:39:46.586900 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:46.586909 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:46.586929 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:46.587233 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:46.587566 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:39:46.587569 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:39:46.587571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:39:46.587739 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:39:46.588197 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:39:46.588206 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:39:46.588500 (+ 294us) client_negotiation.cc:769] Sending connection context
0808 14:39:46.588716 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 14:39:46.589110 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":142,"threads_started":1}
I20260808 14:39:46.591514 31115 heartbeater.cc:347] Connected to a master server at 127.19.144.126:45751
I20260808 14:39:46.591804 31115 heartbeater.cc:464] Registering TS with master...
I20260808 14:39:46.592420 31115 heartbeater.cc:511] Master 127.19.144.126:45751 requested a full tablet report, sending...
I20260808 14:39:46.593490 30648 ts_manager.cc:194] Registered new tserver with Master: 2602bd22316f4b24a8af8a54816c58f5 (127.19.144.67:34539)
I20260808 14:39:46.594129 30648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:42063
I20260808 14:39:46.605029 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:39:46.614956 30688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.607171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59454 (local address 127.19.144.126:45751)
0808 14:39:46.607326 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:46.607329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:46.607537 (+ 208us) server_negotiation.cc:407] Connection header received
0808 14:39:46.607637 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:46.607640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:46.607685 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:46.607765 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:39:46.608512 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.609054 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.609997 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.610256 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.610348 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.611812 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:39:46.611814 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:39:46.611816 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:39:46.611836 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:39:46.613308 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:46.613840 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:46.613867 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:46.613868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:46.613954 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:39:46.614189 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:39:46.614190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:39:46.614191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:39:46.614412 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:39:46.614523 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:46.614703 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 14:39:46.614817 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 14:39:46.618589 30648 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:39:46.621799 30648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:39:46.637187 31129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.630601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56392 (local address 127.19.144.66:43701)
0808 14:39:46.632015 (+ 1414us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:46.632031 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:46.632056 (+ 25us) server_negotiation.cc:407] Connection header received
0808 14:39:46.632141 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:46.632151 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:46.632309 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:46.632475 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:46.632778 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.633993 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.635847 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.636415 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.636536 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.636693 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:46.636954 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 14:39:46.637051 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1301,"thread_start_us":119,"threads_started":1}
I20260808 14:39:46.637179 31130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.630478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:43701 (local address 127.0.0.1:56392)
0808 14:39:46.631184 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:46.631237 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:46.631390 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:46.632467 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:46.632473 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:46.632495 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:46.632656 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:46.632663 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.634154 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.634158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:46.635660 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:46.635672 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.636548 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.636552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:46.636657 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.636695 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 14:39:46.636808 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 14:39:46.637009 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":123,"thread_start_us":97,"threads_started":1}
I20260808 14:39:46.637377 31131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.631333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39168 (local address 127.19.144.65:33333)
0808 14:39:46.631792 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:46.631799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:46.632030 (+ 231us) server_negotiation.cc:407] Connection header received
0808 14:39:46.632133 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:46.632137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:46.632336 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:46.632491 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:46.633065 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.633887 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.635494 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.636443 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.636554 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.636706 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:46.637107 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 14:39:46.637213 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":102,"threads_started":1}
I20260808 14:39:46.637604 31128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.630779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:33333 (local address 127.0.0.1:39168)
0808 14:39:46.631904 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:46.631918 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:46.632029 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:46.632640 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:46.632643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:46.632675 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:46.632890 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:46.632901 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.634062 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.634066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:46.635362 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:46.635373 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.636777 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.636780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:46.636958 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.636983 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 14:39:46.637059 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 14:39:46.637137 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1064,"mutex_wait_us":131,"spinlock_wait_cycles":4352,"thread_start_us":125,"threads_started":1}
I20260808 14:39:46.638999 31127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.630542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:34539 (local address 127.0.0.1:39686)
0808 14:39:46.631780 (+ 1238us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:46.631793 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:46.631894 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:46.632640 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:46.632643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:46.632673 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:46.632916 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:46.632923 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.634311 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.634314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:46.635672 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:46.635679 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.636775 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.636782 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:46.638676 (+ 1894us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.638695 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 14:39:46.638790 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 14:39:46.638859 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1169,"mutex_wait_us":1922,"spinlock_wait_cycles":5760,"thread_start_us":116,"threads_started":1}
I20260808 14:39:46.639098 31132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.631333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39686 (local address 127.19.144.67:34539)
0808 14:39:46.631744 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:46.631758 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:46.632022 (+ 264us) server_negotiation.cc:407] Connection header received
0808 14:39:46.632122 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:46.632127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:46.632336 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:46.632491 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:46.633044 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.634164 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.635786 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.636605 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.636700 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.636978 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:46.638827 (+ 1849us) server_negotiation.cc:299] Negotiation successful
0808 14:39:46.638930 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":119,"threads_started":1}
I20260808 14:39:46.641089 30775 tablet_service.cc:1467] Processing CreateTablet for tablet 02f2b2d976884aa7ae2d03ade392533c (DEFAULT_TABLE table=test-table [id=e6559c93c58e4377a373e8941dc55dc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:46.640947 30911 tablet_service.cc:1467] Processing CreateTablet for tablet 02f2b2d976884aa7ae2d03ade392533c (DEFAULT_TABLE table=test-table [id=e6559c93c58e4377a373e8941dc55dc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:46.641647 31046 tablet_service.cc:1467] Processing CreateTablet for tablet 02f2b2d976884aa7ae2d03ade392533c (DEFAULT_TABLE table=test-table [id=e6559c93c58e4377a373e8941dc55dc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:39:46.642225 30775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02f2b2d976884aa7ae2d03ade392533c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:46.642275 30911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02f2b2d976884aa7ae2d03ade392533c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:46.642678 31046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02f2b2d976884aa7ae2d03ade392533c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:39:46.649245 31133 tablet_bootstrap.cc:492] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5: Bootstrap starting.
I20260808 14:39:46.650722 31135 tablet_bootstrap.cc:492] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144: Bootstrap starting.
I20260808 14:39:46.651789 31133 tablet_bootstrap.cc:654] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:46.653252 31135 tablet_bootstrap.cc:654] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:46.653237 31133 log.cc:824] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:46.653951 31134 tablet_bootstrap.cc:492] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f: Bootstrap starting.
I20260808 14:39:46.654129 31135 log.cc:824] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:46.655514 31133 tablet_bootstrap.cc:492] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5: No bootstrap required, opened a new log
I20260808 14:39:46.655732 31133 ts_tablet_manager.cc:1397] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 14:39:46.656073 31134 tablet_bootstrap.cc:654] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f: Neither blocks nor log segments found. Creating new log.
I20260808 14:39:46.656266 31135 tablet_bootstrap.cc:492] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144: No bootstrap required, opened a new log
I20260808 14:39:46.656454 31135 ts_tablet_manager.cc:1397] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:39:46.656839 31134 log.cc:824] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f: Log is configured to *not* fsync() on all Append() calls
I20260808 14:39:46.658605 31134 tablet_bootstrap.cc:492] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f: No bootstrap required, opened a new log
I20260808 14:39:46.658810 31134 ts_tablet_manager.cc:1397] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 14:39:46.659859 31133 raft_consensus.cc:348] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb9003f93d54d6e8502f3f4556f9144" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43701 } } peers { permanent_uuid: "2602bd22316f4b24a8af8a54816c58f5" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34539 } } peers { permanent_uuid: "4306ba6b42c845f4b1ff6f33ada8bf9f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33333 } }
I20260808 14:39:46.660283 31133 raft_consensus.cc:374] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:46.660440 31133 raft_consensus.cc:729] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2602bd22316f4b24a8af8a54816c58f5, State: Initialized, Role: FOLLOWER
I20260808 14:39:46.660315 31135 raft_consensus.cc:348] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb9003f93d54d6e8502f3f4556f9144" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43701 } } peers { permanent_uuid: "2602bd22316f4b24a8af8a54816c58f5" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34539 } } peers { permanent_uuid: "4306ba6b42c845f4b1ff6f33ada8bf9f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33333 } }
I20260808 14:39:46.660606 31135 raft_consensus.cc:374] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:46.660683 31135 raft_consensus.cc:729] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb9003f93d54d6e8502f3f4556f9144, State: Initialized, Role: FOLLOWER
I20260808 14:39:46.660979 31133 consensus_queue.cc:260] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb9003f93d54d6e8502f3f4556f9144" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43701 } } peers { permanent_uuid: "2602bd22316f4b24a8af8a54816c58f5" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34539 } } peers { permanent_uuid: "4306ba6b42c845f4b1ff6f33ada8bf9f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33333 } }
I20260808 14:39:46.661188 31135 consensus_queue.cc:260] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb9003f93d54d6e8502f3f4556f9144" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43701 } } peers { permanent_uuid: "2602bd22316f4b24a8af8a54816c58f5" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34539 } } peers { permanent_uuid: "4306ba6b42c845f4b1ff6f33ada8bf9f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33333 } }
I20260808 14:39:46.661842 31115 heartbeater.cc:503] Master 127.19.144.126:45751 was elected leader, sending a full tablet report...
I20260808 14:39:46.662482 31135 ts_tablet_manager.cc:1428] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:39:46.662858 30978 heartbeater.cc:503] Master 127.19.144.126:45751 was elected leader, sending a full tablet report...
I20260808 14:39:46.663498 31133 ts_tablet_manager.cc:1428] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260808 14:39:46.663525 31134 raft_consensus.cc:348] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb9003f93d54d6e8502f3f4556f9144" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43701 } } peers { permanent_uuid: "2602bd22316f4b24a8af8a54816c58f5" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34539 } } peers { permanent_uuid: "4306ba6b42c845f4b1ff6f33ada8bf9f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33333 } }
I20260808 14:39:46.663832 31134 raft_consensus.cc:374] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:39:46.663900 31134 raft_consensus.cc:729] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4306ba6b42c845f4b1ff6f33ada8bf9f, State: Initialized, Role: FOLLOWER
I20260808 14:39:46.664388 31134 consensus_queue.cc:260] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb9003f93d54d6e8502f3f4556f9144" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43701 } } peers { permanent_uuid: "2602bd22316f4b24a8af8a54816c58f5" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34539 } } peers { permanent_uuid: "4306ba6b42c845f4b1ff6f33ada8bf9f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33333 } }
I20260808 14:39:46.665194 30843 heartbeater.cc:503] Master 127.19.144.126:45751 was elected leader, sending a full tablet report...
I20260808 14:39:46.665426 31134 ts_tablet_manager.cc:1428] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 14:39:46.795614 31139 raft_consensus.cc:482] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:39:46.795935 31139 raft_consensus.cc:504] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb9003f93d54d6e8502f3f4556f9144" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43701 } } peers { permanent_uuid: "2602bd22316f4b24a8af8a54816c58f5" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34539 } } peers { permanent_uuid: "4306ba6b42c845f4b1ff6f33ada8bf9f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33333 } }
I20260808 14:39:46.797297 31139 leader_election.cc:288] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bb9003f93d54d6e8502f3f4556f9144 (127.19.144.66:43701), 4306ba6b42c845f4b1ff6f33ada8bf9f (127.19.144.65:33333)
I20260808 14:39:46.803062 31131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.797836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:39767 (local address 127.19.144.65:33333)
0808 14:39:46.797996 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:46.798003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:46.798237 (+ 234us) server_negotiation.cc:407] Connection header received
0808 14:39:46.798305 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:46.798307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:46.798367 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:46.798464 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:46.798824 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.799629 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.801585 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.802240 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.802374 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.802447 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:46.802838 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 14:39:46.802914 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 14:39:46.803062 31129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.797716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:50651 (local address 127.19.144.66:43701)
0808 14:39:46.797875 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:46.797882 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:46.797905 (+ 23us) server_negotiation.cc:407] Connection header received
0808 14:39:46.798235 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:46.798239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:46.798301 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:46.798400 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:39:46.798825 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.799630 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.801585 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.802240 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.802374 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.802431 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:46.802833 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 14:39:46.802914 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 14:39:46.803093 31118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.797565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:43701 (local address 127.19.144.67:50651)
0808 14:39:46.797815 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:46.797831 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:46.797942 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:46.798442 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:46.798446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:46.798465 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:46.798696 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:46.798701 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.799785 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.799790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:46.801419 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:46.801427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.802392 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.802396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:46.802641 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.802694 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 14:39:46.802787 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 14:39:46.802921 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":454,"spinlock_wait_cycles":14336}
I20260808 14:39:46.803102 31142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.797703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:33333 (local address 127.19.144.67:39767)
0808 14:39:46.798114 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:46.798126 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:39:46.798220 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:39:46.798468 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:39:46.798470 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:39:46.798482 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:39:46.798684 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:46.798694 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.799785 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.799790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:46.801413 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:39:46.801427 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.802392 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.802396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:39:46.802612 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.802685 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 14:39:46.802809 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 14:39:46.802916 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":189,"thread_start_us":187,"threads_started":1}
I20260808 14:39:46.803947 30932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f2b2d976884aa7ae2d03ade392533c" candidate_uuid: "2602bd22316f4b24a8af8a54816c58f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bb9003f93d54d6e8502f3f4556f9144" is_pre_election: true
I20260808 14:39:46.804000 30796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f2b2d976884aa7ae2d03ade392533c" candidate_uuid: "2602bd22316f4b24a8af8a54816c58f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4306ba6b42c845f4b1ff6f33ada8bf9f" is_pre_election: true
I20260808 14:39:46.804317 30932 raft_consensus.cc:2454] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2602bd22316f4b24a8af8a54816c58f5 in term 0.
I20260808 14:39:46.804323 30796 raft_consensus.cc:2454] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2602bd22316f4b24a8af8a54816c58f5 in term 0.
I20260808 14:39:46.804893 31000 leader_election.cc:302] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2602bd22316f4b24a8af8a54816c58f5, 4306ba6b42c845f4b1ff6f33ada8bf9f, 6bb9003f93d54d6e8502f3f4556f9144; no voters:
I20260808 14:39:46.805238 31139 raft_consensus.cc:2781] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:39:46.805332 31139 raft_consensus.cc:482] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:39:46.805378 31139 raft_consensus.cc:3037] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:46.806644 31139 raft_consensus.cc:504] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb9003f93d54d6e8502f3f4556f9144" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43701 } } peers { permanent_uuid: "2602bd22316f4b24a8af8a54816c58f5" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34539 } } peers { permanent_uuid: "4306ba6b42c845f4b1ff6f33ada8bf9f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33333 } }
I20260808 14:39:46.807062 31139 leader_election.cc:288] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [CANDIDATE]: Term 1 election: Requested vote from peers 6bb9003f93d54d6e8502f3f4556f9144 (127.19.144.66:43701), 4306ba6b42c845f4b1ff6f33ada8bf9f (127.19.144.65:33333)
I20260808 14:39:46.807432 30932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f2b2d976884aa7ae2d03ade392533c" candidate_uuid: "2602bd22316f4b24a8af8a54816c58f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bb9003f93d54d6e8502f3f4556f9144"
I20260808 14:39:46.807523 30796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f2b2d976884aa7ae2d03ade392533c" candidate_uuid: "2602bd22316f4b24a8af8a54816c58f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4306ba6b42c845f4b1ff6f33ada8bf9f"
I20260808 14:39:46.807583 30932 raft_consensus.cc:3037] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:46.807687 30796 raft_consensus.cc:3037] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:39:46.808777 30932 raft_consensus.cc:2454] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2602bd22316f4b24a8af8a54816c58f5 in term 1.
I20260808 14:39:46.808799 30796 raft_consensus.cc:2454] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2602bd22316f4b24a8af8a54816c58f5 in term 1.
I20260808 14:39:46.809136 31002 leader_election.cc:302] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2602bd22316f4b24a8af8a54816c58f5, 4306ba6b42c845f4b1ff6f33ada8bf9f; no voters:
I20260808 14:39:46.809350 31139 raft_consensus.cc:2781] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:39:46.809656 31139 raft_consensus.cc:686] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [term 1 LEADER]: Becoming Leader. State: Replica: 2602bd22316f4b24a8af8a54816c58f5, State: Running, Role: LEADER
I20260808 14:39:46.810045 31139 consensus_queue.cc:237] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb9003f93d54d6e8502f3f4556f9144" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43701 } } peers { permanent_uuid: "2602bd22316f4b24a8af8a54816c58f5" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34539 } } peers { permanent_uuid: "4306ba6b42c845f4b1ff6f33ada8bf9f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33333 } }
I20260808 14:39:46.813522 30648 catalog_manager.cc:5495] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2602bd22316f4b24a8af8a54816c58f5 (127.19.144.67). New cstate: current_term: 1 leader_uuid: "2602bd22316f4b24a8af8a54816c58f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb9003f93d54d6e8502f3f4556f9144" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2602bd22316f4b24a8af8a54816c58f5" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 34539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4306ba6b42c845f4b1ff6f33ada8bf9f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33333 } health_report { overall_health: UNKNOWN } } }
W20260808 14:39:46.817531 30844 tablet.cc:2369] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:39:46.824582 31116 tablet.cc:2369] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:39:46.827950 31132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.824008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39692 (local address 127.19.144.67:34539)
0808 14:39:46.824152 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 14:39:46.824158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:39:46.824176 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:39:46.824412 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:39:46.824416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:39:46.824493 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:39:46.824611 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:39:46.824944 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.825554 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.826481 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:39:46.826696 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:39:46.826791 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:39:46.827130 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:39:46.827576 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:39:46.827674 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 14:39:46.827772 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 14:39:46.827829 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:39:46.834357 30796 raft_consensus.cc:1261] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f [term 1 FOLLOWER]: Refusing update from remote peer 2602bd22316f4b24a8af8a54816c58f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:46.834357 30932 raft_consensus.cc:1261] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144 [term 1 FOLLOWER]: Refusing update from remote peer 2602bd22316f4b24a8af8a54816c58f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:39:46.835153 31143 consensus_queue.cc:1035] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bb9003f93d54d6e8502f3f4556f9144" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 43701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:39:46.835342 31139 consensus_queue.cc:1035] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4306ba6b42c845f4b1ff6f33ada8bf9f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 14:39:46.836561 30979 tablet.cc:2369] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:39:46.844529 31148 mvcc.cc:205] Tried to move back new op lower bound from 7316275146065530880 to 7316275145976696832. Current Snapshot: MvccSnapshot[applied={T|T < 7316275146065530880}]
I20260808 14:39:46.845395 31147 mvcc.cc:205] Tried to move back new op lower bound from 7316275146065530880 to 7316275145976696832. Current Snapshot: MvccSnapshot[applied={T|T < 7316275146065530880}]
I20260808 14:39:46.852898 30648 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:46.853111 30648 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:39:46.855690 30648 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c7cea28395b4160b8cfb3d39fd1855d: Sending DeleteTablet for 3 replicas of tablet 02f2b2d976884aa7ae2d03ade392533c
I20260808 14:39:46.856566 30911 tablet_service.cc:1514] Processing DeleteTablet for tablet 02f2b2d976884aa7ae2d03ade392533c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:46 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:56392
I20260808 14:39:46.856616 30775 tablet_service.cc:1514] Processing DeleteTablet for tablet 02f2b2d976884aa7ae2d03ade392533c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:46 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:39168
I20260808 14:39:46.856853 31046 tablet_service.cc:1514] Processing DeleteTablet for tablet 02f2b2d976884aa7ae2d03ade392533c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:39:46 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:39686
I20260808 14:39:46.857295 31157 tablet_replica.cc:330] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f: stopping tablet replica
I20260808 14:39:46.857344 31158 tablet_replica.cc:330] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5: stopping tablet replica
I20260808 14:39:46.857713 31158 raft_consensus.cc:2229] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:39:46.857712 31157 raft_consensus.cc:2229] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:46.858045 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 30711
I20260808 14:39:46.858073 31157 raft_consensus.cc:2258] T 02f2b2d976884aa7ae2d03ade392533c P 4306ba6b42c845f4b1ff6f33ada8bf9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:46.858707 31156 tablet_replica.cc:330] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144: stopping tablet replica
I20260808 14:39:46.859042 31156 raft_consensus.cc:2229] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:39:46.859301 31156 raft_consensus.cc:2258] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:46.860450 31158 raft_consensus.cc:2258] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:39:46.862032 31158 ts_tablet_manager.cc:1905] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:46.862033 31156 ts_tablet_manager.cc:1905] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:39:46.864974 31158 ts_tablet_manager.cc:1918] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:46.864974 31156 ts_tablet_manager.cc:1918] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:39:46.865091 31156 log.cc:1196] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/02f2b2d976884aa7ae2d03ade392533c
I20260808 14:39:46.865207 31158 log.cc:1196] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/02f2b2d976884aa7ae2d03ade392533c
I20260808 14:39:46.865456 31156 ts_tablet_manager.cc:1939] T 02f2b2d976884aa7ae2d03ade392533c P 6bb9003f93d54d6e8502f3f4556f9144: Deleting consensus metadata
I20260808 14:39:46.865707 31158 ts_tablet_manager.cc:1939] T 02f2b2d976884aa7ae2d03ade392533c P 2602bd22316f4b24a8af8a54816c58f5: Deleting consensus metadata
I20260808 14:39:46.867004 30632 catalog_manager.cc:4841] TS 6bb9003f93d54d6e8502f3f4556f9144 (127.19.144.66:43701): tablet 02f2b2d976884aa7ae2d03ade392533c (table test-table [id=e6559c93c58e4377a373e8941dc55dc9]) successfully deleted
I20260808 14:39:46.867012 30633 catalog_manager.cc:4841] TS 2602bd22316f4b24a8af8a54816c58f5 (127.19.144.67:34539): tablet 02f2b2d976884aa7ae2d03ade392533c (table test-table [id=e6559c93c58e4377a373e8941dc55dc9]) successfully deleted
W20260808 14:39:46.868875 30634 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.144.65:33333 (error 108)
I20260808 14:39:46.869895 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 30848
I20260808 14:39:46.870193 31127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:39:46.869862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:33333 (local address 127.0.0.1:39170)
0808 14:39:46.870029 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 14:39:46.870103 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260808 14:39:46.870975 30634 catalog_manager.cc:4584] TS 4306ba6b42c845f4b1ff6f33ada8bf9f (127.19.144.65:33333): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 02f2b2d976884aa7ae2d03ade392533c: Network error: Client connection negotiation failed: client connection to 127.19.144.65:33333: connect: Connection refused (error 111)
I20260808 14:39:46.877703 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 30983
I20260808 14:39:46.884941 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 30614
2026-08-08T14:39:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2598 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:39:46 dist-test-slave-p3t5 krb5kdc[31162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:39:46 dist-test-slave-p3t5 krb5kdc[31162](info): set up 2 sockets
Aug 08 14:39:46 dist-test-slave-p3t5 krb5kdc[31162](info): commencing operation
krb5kdc: starting...
W20260808 14:39:47.964658 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:39:48 dist-test-slave-p3t5 krb5kdc[31162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786199988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:39:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:39:48Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.144.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.144.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.144.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/rangeradmin_127.19.144.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.144.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/rangeradmin_127.19.144.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.144.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.144.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.144.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/rangerlookup_127.19.144.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.144.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/rangerlookup_127.19.144.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.144.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.144.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.144.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/HTTP_127.19.144.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.144.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/HTTP_127.19.144.84@KRBTEST.COM.keytab.
I20260808 14:39:48.156366 20033 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskCQkDbp/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/postgres -l logfile start
2026-08-08 14:39:50.814 UTC [31194] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 14:39:50.814 UTC [31194] LOG: listening on IPv4 address "127.19.144.84", port 48753
2026-08-08 14:39:50.818 UTC [31194] LOG: listening on Unix socket "/tmp/.s.PGSQL.48753"
2026-08-08 14:39:50.841 UTC [31197] LOG: database system was shut down at 2026-08-08 14:39:49 UTC
2026-08-08 14:39:50.845 UTC [31194] LOG: database system is ready to accept connections
I20260808 14:39:51.825114 20033 mini_postgres.cc:96] Postgres bound to 48753
2026-08-08 14:39:51.835 UTC [31205] FATAL: database "slave" does not exist
127.19.144.84:48753 - accepting connections
I20260808 14:39:51.836684 20033 mini_ranger.cc:161] Starting Ranger...
I20260808 14:39:51.860080 20033 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 14:39:52.383860 20033 mini_ranger.cc:87] Created ranger Postgres database
I20260808 14:39:52.383996 20033 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-admin
2026-08-08 14:39:52,722 [I] DB FLAVOR :POSTGRES
2026-08-08 14:39:52,722 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 14:39:52,722 [I] Checking connection..
2026-08-08 14:39:52,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 14:39:53,018 [I] Checking connection passed.
2026-08-08 14:39:53,018 [I] --------- Verifying version history table ---------
2026-08-08 14:39:53,018 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 14:39:53,281 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 14:39:53,281 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 14:39:53,532 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 14:39:53,532 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 14:39:53,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 14:39:53.776 UTC [31272] WARNING: there is no transaction in progress
2026-08-08 14:39:53,784 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 14:39:53,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 14:39:54,027 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 14:39:54,028 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 14:39:54,028 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 14:39:54,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 14:39:54,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-p3t5.c.gcp-upstream.internal','N') ;"
2026-08-08 14:39:54,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 14:39:55,038 [I] Table x_portal_user does not exist in database ranger
2026-08-08 14:39:55,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 14:39:55,288 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 14:39:55,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 14:39:55,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 14:39:55,846 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 14:39:55,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 14:39:56.919 UTC [31384] WARNING: there is no transaction in progress
2026-08-08 14:39:56.938 UTC [31384] WARNING: there is no transaction in progress
2026-08-08 14:39:56.958 UTC [31384] WARNING: there is no transaction in progress
2026-08-08 14:39:56.975 UTC [31384] WARNING: there is no transaction in progress
2026-08-08 14:39:56.999 UTC [31384] WARNING: there is no transaction in progress
2026-08-08 14:39:57.019 UTC [31384] WARNING: there is no transaction in progress
2026-08-08 14:39:57.081 UTC [31384] WARNING: there is no transaction in progress
2026-08-08 14:39:57.090 UTC [31384] WARNING: there is no transaction in progress
2026-08-08 14:39:57.106 UTC [31384] WARNING: there is no transaction in progress
2026-08-08 14:39:57.117 UTC [31384] WARNING: there is no transaction in progress
2026-08-08 14:39:57.129 UTC [31384] WARNING: there is no transaction in progress
2026-08-08 14:39:57.144 UTC [31384] WARNING: there is no transaction in progress
2026-08-08 14:39:57.154 UTC [31384] WARNING: there is no transaction in progress
2026-08-08 14:39:57.161 UTC [31384] WARNING: there is no transaction in progress
2026-08-08 14:39:57.170 UTC [31384] WARNING: there is no transaction in progress
2026-08-08 14:39:57.178 UTC [31384] WARNING: there is no transaction in progress
2026-08-08 14:39:57,926 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 14:39:57,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 14:39:58,172 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 14:39:58,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 14:39:58,450 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 14:39:58,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 14:39:58,721 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 14:39:58,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 14:39:59,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 14:39:59,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-p3t5.c.gcp-upstream.internal';"
2026-08-08 14:39:59,539 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 14:39:59,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 14:39:59,803 [I] DB_PATCHES have already been applied
I20260808 14:39:59.810294 20033 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-admin:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskCQkDbp/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 14:39:59.810381 20033 mini_ranger.cc:193] Using host: 127.19.144.84
I20260808 14:39:59.815356 20033 mini_ranger.cc:227] Ranger admin URL: http://127.19.144.84:54475
Aug 08, 2026 2:40:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:40:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:40:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 2:40:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 2:40:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 2:40:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 2:40:00 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 14:40:00.962278 20033 mini_ranger.cc:160] Time spent starting Ranger: real 9.126s user 0.000s sys 0.014s
Aug 08, 2026 2:40:21 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 2:40:21 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 2:40:21 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 2:40:21 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 2:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 2:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 2:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 2:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 2:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 2:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 2:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 2:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 2:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 2:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 2:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 2:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 2:40:21 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 2:40:22 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 14:40:22.987370 20033 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.144.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.144.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.144.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/rangerkms_127.19.144.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.144.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/rangerkms_127.19.144.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.144.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.144.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.144.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/HTTP_127.19.144.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.144.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/HTTP_127.19.144.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 14:40:23 dist-test-slave-p3t5 krb5kdc[31162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200023, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 14:40:23.101157 20033 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 14:40:23.112695 20033 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 14:40:24.001592 20033 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 14:40:24.001734 20033 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-kms
I20260808 14:40:24.001842 20033 mini_ranger_kms.cc:227] Using postgres at 127.19.144.84:48753
2026-08-08 14:40:24,731 [I] DB FLAVOR :POSTGRES
2026-08-08 14:40:24,732 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 14:40:24,732 [I] Checking connection
2026-08-08 14:40:24,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 14:40:25,010 [I] connection success
2026-08-08 14:40:25,011 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 14:40:25,011 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 14:40:25,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 14:40:25,269 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 14:40:25,269 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 14:40:25,269 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 14:40:25,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:48753/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 14:40:25,549 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 14:40:26.078064 20033 mini_ranger_kms.cc:310] Created kms service
I20260808 14:40:26.289829 20033 mini_ranger_kms.cc:326] Created kudu user
I20260808 14:40:26.337989 20033 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 14:40:26 dist-test-slave-p3t5 krb5kdc[31162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200023, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
I20260808 14:40:26.820389 20033 mini_ranger_kms.cc:384] Added ranger policy
I20260808 14:40:26.820562 20033 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-kms:/tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 14:40:26.820585 20033 mini_ranger_kms.cc:240] Using host: 127.19.144.84
I20260808 14:40:26.825560 20033 mini_ranger_kms.cc:276] Ranger KMS PID: 31570
I20260808 14:40:26.825752 20033 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.144.84:55315
Aug 08, 2026 2:40:27 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:40:27 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:40:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 2:40:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 2:40:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 2:40:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 14:40:29.088497 20033 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.987s user 0.015s sys 0.260s
I20260808 14:40:29.088642 20033 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 14:40:29.088750 20033 mini_ranger_kms.cc:401] 127.19.144.84:55315/kms/v1/keys
Aug 08 14:40:29 dist-test-slave-p3t5 krb5kdc[31162](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786200029, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.144.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 14:40:29 dist-test-slave-p3t5 krb5kdc[31162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200029, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.144.84@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
Aug 08 14:40:29 dist-test-slave-p3t5 krb5kdc[31162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200029, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.144.84@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
Aug 08 14:40:32 dist-test-slave-p3t5 krb5kdc[31162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:40:32.287446 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:34645
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:33579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:34645
--ranger_config_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.144.84:55315/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:40:32.437484 31622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:40:32.437846 31622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:40:32.437917 31622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:40:32.442039 31622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:40:32.442111 31622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:40:32.442131 31622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:40:32.442148 31622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:40:32.442163 31622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:40:32.448237 31622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:33579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.144.84:55315/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:34645
--ranger_config_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:34645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:40:32.449522 31622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:40:32.450680 31622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:40:32.460566 31627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:40:32.460599 31628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:40:32.460599 31630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:40:32.461022 31622 server_base.cc:1034] running on GCE node
I20260808 14:40:32.461802 31622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:40:32.463135 31622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:40:32.464368 31622 hybrid_clock.cc:648] HybridClock initialized: now 1786200032464348 us; error 41 us; skew 500 ppm
Aug 08 14:40:32 dist-test-slave-p3t5 krb5kdc[31162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200032, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:40:32.467676 31622 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:40:32.468729 31622 webserver.cc:460] Webserver started at http://127.19.144.126:36179/ using document root <none> and password file <none>
I20260808 14:40:32.469365 31622 fs_manager.cc:359] Metadata directory not provided
I20260808 14:40:32.469455 31622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:40:32.469762 31622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 14:40:32 dist-test-slave-p3t5 krb5kdc[31162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200032, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
I20260808 14:40:32.538153 31622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "0130f457890a463c880d078156bbaa38"
format_stamp: "Formatted at 2026-08-08 14:40:32 on dist-test-slave-p3t5"
server_key: "f8faf63abde8afe4bee0bac5a519ae5c"
server_key_iv: "ec7f8325635b3ee7098b9b580e8886f0"
server_key_version: "kuduclusterkey@0"
I20260808 14:40:32.538805 31622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "0130f457890a463c880d078156bbaa38"
format_stamp: "Formatted at 2026-08-08 14:40:32 on dist-test-slave-p3t5"
server_key: "f8faf63abde8afe4bee0bac5a519ae5c"
server_key_iv: "ec7f8325635b3ee7098b9b580e8886f0"
server_key_version: "kuduclusterkey@0"
I20260808 14:40:32.543206 31622 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:40:32.564917 31637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:40:32.567876 31622 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260808 14:40:32.568073 31622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "0130f457890a463c880d078156bbaa38"
format_stamp: "Formatted at 2026-08-08 14:40:32 on dist-test-slave-p3t5"
server_key: "f8faf63abde8afe4bee0bac5a519ae5c"
server_key_iv: "ec7f8325635b3ee7098b9b580e8886f0"
server_key_version: "kuduclusterkey@0"
I20260808 14:40:32.568243 31622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:40:32.584127 31622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:40:32.587702 31622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:40:32.587945 31622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:40:32.596776 31622 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:34645
I20260808 14:40:32.596827 31689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:34645 every 8 connection(s)
I20260808 14:40:32.597992 31622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:40:32.601966 31690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:40:32.608098 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 31622
I20260808 14:40:32.608259 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:40:32.608558 20033 external_mini_cluster.cc:1428] Setting key d2d0dc1097c285ce94ca90ef8f338476
I20260808 14:40:32.610450 31690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38: Bootstrap starting.
I20260808 14:40:32.613554 31690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38: Neither blocks nor log segments found. Creating new log.
I20260808 14:40:32.614605 31690 log.cc:824] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38: Log is configured to *not* fsync() on all Append() calls
I20260808 14:40:32.617301 31690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38: No bootstrap required, opened a new log
Aug 08 14:40:32 dist-test-slave-p3t5 krb5kdc[31162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:40:32.622596 31690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0130f457890a463c880d078156bbaa38" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34645 } }
I20260808 14:40:32.623044 31690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:40:32.623191 31690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0130f457890a463c880d078156bbaa38, State: Initialized, Role: FOLLOWER
I20260808 14:40:32.623854 31690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0130f457890a463c880d078156bbaa38" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34645 } }
I20260808 14:40:32.624045 31690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:40:32.624145 31690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:40:32.624253 31690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:40:32.624640 31694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:32.610201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42590 (local address 127.19.144.126:34645)
0808 14:40:32.610764 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0808 14:40:32.610787 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 14:40:32.610837 (+ 50us) server_negotiation.cc:407] Connection header received
0808 14:40:32.611511 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:40:32.611528 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:40:32.611871 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:40:32.612238 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:40:32.613189 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:32.614598 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:32.615763 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:32.616276 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:32.616451 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:32.619324 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:40:32.619330 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:40:32.619345 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:40:32.619375 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:40:32.621245 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:40:32.621819 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:40:32.621826 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:40:32.621832 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:40:32.621976 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:40:32.622306 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:40:32.622309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:40:32.622310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:40:32.622765 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:40:32.623014 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 14:40:32.623281 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 14:40:32.623642 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":181,"threads_started":1}
I20260808 14:40:32.625531 31690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0130f457890a463c880d078156bbaa38" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34645 } }
I20260808 14:40:32.625957 31690 leader_election.cc:302] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0130f457890a463c880d078156bbaa38; no voters:
I20260808 14:40:32.626318 31690 leader_election.cc:288] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:40:32.626482 31696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:40:32.626770 31696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [term 1 LEADER]: Becoming Leader. State: Replica: 0130f457890a463c880d078156bbaa38, State: Running, Role: LEADER
I20260808 14:40:32.627414 31696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0130f457890a463c880d078156bbaa38" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34645 } }
I20260808 14:40:32.627815 31690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:40:32.628645 31696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0130f457890a463c880d078156bbaa38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0130f457890a463c880d078156bbaa38" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34645 } } }
I20260808 14:40:32.629011 31696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [sys.catalog]: This master's current role is: LEADER
I20260808 14:40:32.629324 31696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0130f457890a463c880d078156bbaa38. Latest consensus state: current_term: 1 leader_uuid: "0130f457890a463c880d078156bbaa38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0130f457890a463c880d078156bbaa38" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34645 } } }
I20260808 14:40:32.629479 31696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38 [sys.catalog]: This master's current role is: LEADER
I20260808 14:40:32.629832 31705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:40:32.630227 31690 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 14:40:32.636325 31705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:40:32.648814 31705 catalog_manager.cc:1347] Generated new cluster ID: 40b63118793d4ad8ae4b7b15557650f0
I20260808 14:40:32.648896 31705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:40:32.665570 31705 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:40:32.667284 31705 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:40:32.678705 31705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38: Generated new TSK 0
I20260808 14:40:32.679852 31705 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:40:33.421264 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.144.84:55315/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34645
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:33579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:40:33.576315 31738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:40:33.576654 31738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:40:33.576776 31738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:40:33.581923 31738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:40:33.582074 31738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:40:33.582195 31738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:40:33.588066 31738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:33579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.144.84:55315/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:40:33.589349 31738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:40:33.590582 31738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:40:33.602180 31748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:40:33.602123 31745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:40:33.602591 31746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:40:33.602798 31738 server_base.cc:1034] running on GCE node
I20260808 14:40:33.603281 31738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:40:33.604112 31738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:40:33.605300 31738 hybrid_clock.cc:648] HybridClock initialized: now 1786200033605271 us; error 47 us; skew 500 ppm
Aug 08 14:40:33 dist-test-slave-p3t5 krb5kdc[31162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200033, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:40:33.609144 31738 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:40:33.610432 31738 webserver.cc:460] Webserver started at http://127.19.144.65:37717/ using document root <none> and password file <none>
I20260808 14:40:33.611240 31738 fs_manager.cc:359] Metadata directory not provided
I20260808 14:40:33.611344 31738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:40:33.611640 31738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 14:40:33 dist-test-slave-p3t5 krb5kdc[31162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200033, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
I20260808 14:40:33.629367 31738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "42b3be04b51c4114ac973d8136b70a74"
format_stamp: "Formatted at 2026-08-08 14:40:33 on dist-test-slave-p3t5"
server_key: "6098b384c8185e1601e3083275e69d4c"
server_key_iv: "3de21f819ee1fa963977022865f27b99"
server_key_version: "kuduclusterkey@0"
I20260808 14:40:33.633714 31738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "42b3be04b51c4114ac973d8136b70a74"
format_stamp: "Formatted at 2026-08-08 14:40:33 on dist-test-slave-p3t5"
server_key: "6098b384c8185e1601e3083275e69d4c"
server_key_iv: "3de21f819ee1fa963977022865f27b99"
server_key_version: "kuduclusterkey@0"
I20260808 14:40:33.639686 31738 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 14:40:33.666404 31755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:40:33.668557 31738 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260808 14:40:33.668776 31738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "42b3be04b51c4114ac973d8136b70a74"
format_stamp: "Formatted at 2026-08-08 14:40:33 on dist-test-slave-p3t5"
server_key: "6098b384c8185e1601e3083275e69d4c"
server_key_iv: "3de21f819ee1fa963977022865f27b99"
server_key_version: "kuduclusterkey@0"
I20260808 14:40:33.669059 31738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:40:33.696658 31738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:40:33.702482 31738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:40:33.702693 31738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:40:33.703421 31738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:40:33.704515 31738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:40:33.704594 31738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:40:33.704658 31738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:40:33.704685 31738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:40:33.726739 31738 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:43111
I20260808 14:40:33.728067 31738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:40:33.731518 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 31738
I20260808 14:40:33.731660 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:40:33.732007 20033 external_mini_cluster.cc:1428] Setting key 4ab299aee232743c2bc922185fccb766
I20260808 14:40:33.733259 31873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:43111 every 8 connection(s)
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
Aug 08 14:40:33 dist-test-slave-p3t5 krb5kdc[31162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200033, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
I20260808 14:40:33.768939 31878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:33.738706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:34645 (local address 127.19.144.65:45083)
0808 14:40:33.740044 (+ 1338us) negotiation.cc:107] Waiting for socket to connect
0808 14:40:33.740107 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 14:40:33.741176 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:40:33.746512 (+ 5336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:40:33.746527 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:40:33.749249 (+ 2722us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:40:33.750147 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:40:33.750178 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:40:33.751682 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:40:33.751687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:40:33.753312 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:40:33.753330 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:40:33.754089 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:40:33.754094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:40:33.754311 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:33.755076 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:40:33.755099 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:40:33.761644 (+ 6545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:40:33.764655 (+ 3011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:40:33.764664 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:40:33.764687 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:40:33.765168 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:40:33.765575 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:40:33.765581 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:40:33.765586 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:40:33.765849 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:40:33.766508 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:40:33.766519 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:40:33.766939 (+ 420us) client_negotiation.cc:769] Sending connection context
0808 14:40:33.767194 (+ 255us) client_negotiation.cc:240] Negotiation successful
0808 14:40:33.767650 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1096,"thread_start_us":165,"threads_started":1}
I20260808 14:40:33.770076 31877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:33.738141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:45083 (local address 127.19.144.126:34645)
0808 14:40:33.745890 (+ 7749us) server_negotiation.cc:206] Beginning negotiation
0808 14:40:33.745896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:40:33.745917 (+ 21us) server_negotiation.cc:407] Connection header received
0808 14:40:33.745977 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:40:33.745981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:40:33.746049 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:40:33.750377 (+ 4328us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:40:33.750533 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:33.751492 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:33.753511 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:33.753887 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:33.754694 (+ 807us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:33.761951 (+ 7257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:40:33.761958 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:40:33.761964 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:40:33.762008 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:40:33.764463 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:40:33.765326 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:40:33.765333 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:40:33.765337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:40:33.765453 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:40:33.765980 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:40:33.765986 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:40:33.765991 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:40:33.766344 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:40:33.766447 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 14:40:33.769638 (+ 3191us) server_negotiation.cc:299] Negotiation successful
0808 14:40:33.769818 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7617,"thread_start_us":92,"threads_started":1}
I20260808 14:40:33.771435 31874 heartbeater.cc:347] Connected to a master server at 127.19.144.126:34645
I20260808 14:40:33.771725 31874 heartbeater.cc:464] Registering TS with master...
I20260808 14:40:33.772428 31874 heartbeater.cc:511] Master 127.19.144.126:34645 requested a full tablet report, sending...
I20260808 14:40:33.774686 31654 ts_manager.cc:194] Registered new tserver with Master: 42b3be04b51c4114ac973d8136b70a74 (127.19.144.65:43111)
Aug 08 14:40:33 dist-test-slave-p3t5 krb5kdc[31162](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786200033, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:40:33.776549 31654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:45083
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:40:33.783306 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.144.84:55315/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34645
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:33579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:40:33.956022 31880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:40:33.956426 31880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:40:33.956494 31880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:40:33.962837 31880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:40:33.962944 31880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:40:33.963033 31880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:40:33.971318 31880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:33579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.144.84:55315/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:40:33.972862 31880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:40:33.974368 31880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:40:33.988463 31889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:40:33.988930 31887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:40:33.989804 31886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:40:33.990078 31880 server_base.cc:1034] running on GCE node
I20260808 14:40:33.990797 31880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:40:33.991658 31880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:40:33.992846 31880 hybrid_clock.cc:648] HybridClock initialized: now 1786200033992771 us; error 102 us; skew 500 ppm
Aug 08 14:40:33 dist-test-slave-p3t5 krb5kdc[31162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200033, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:40:33.997026 31880 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:40:33.998275 31880 webserver.cc:460] Webserver started at http://127.19.144.66:42931/ using document root <none> and password file <none>
I20260808 14:40:33.999024 31880 fs_manager.cc:359] Metadata directory not provided
I20260808 14:40:33.999197 31880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:40:33.999507 31880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 14:40:33 dist-test-slave-p3t5 krb5kdc[31162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200033, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
I20260808 14:40:34.016875 31880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "35c14622489a4cffaf32b9afc4403d97"
format_stamp: "Formatted at 2026-08-08 14:40:34 on dist-test-slave-p3t5"
server_key: "fbc7f71e6d0b83f6e75a23cae95fec1c"
server_key_iv: "c397e34f8cc7fd999238106d28580b53"
server_key_version: "kuduclusterkey@0"
I20260808 14:40:34.017802 31880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "35c14622489a4cffaf32b9afc4403d97"
format_stamp: "Formatted at 2026-08-08 14:40:34 on dist-test-slave-p3t5"
server_key: "fbc7f71e6d0b83f6e75a23cae95fec1c"
server_key_iv: "c397e34f8cc7fd999238106d28580b53"
server_key_version: "kuduclusterkey@0"
I20260808 14:40:34.023018 31880 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 14:40:34.045158 31896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:40:34.046921 31880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:40:34.047112 31880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "35c14622489a4cffaf32b9afc4403d97"
format_stamp: "Formatted at 2026-08-08 14:40:34 on dist-test-slave-p3t5"
server_key: "fbc7f71e6d0b83f6e75a23cae95fec1c"
server_key_iv: "c397e34f8cc7fd999238106d28580b53"
server_key_version: "kuduclusterkey@0"
I20260808 14:40:34.047291 31880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:40:34.065827 31880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:40:34.069890 31880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:40:34.070163 31880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:40:34.071013 31880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:40:34.072398 31880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:40:34.072490 31880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:40:34.072568 31880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:40:34.072679 31880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:40:34.084971 31880 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:37973
I20260808 14:40:34.086414 31880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:40:34.088106 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 31880
I20260808 14:40:34.088243 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:40:34.088554 20033 external_mini_cluster.cc:1428] Setting key d1eddd344721a9dccd7009e0c375c636
I20260808 14:40:34.094604 32011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:37973 every 8 connection(s)
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
Aug 08 14:40:34 dist-test-slave-p3t5 krb5kdc[31162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200033, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:40:34.122186 31877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.103218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:57721 (local address 127.19.144.126:34645)
0808 14:40:34.103395 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 14:40:34.103402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:40:34.104704 (+ 1302us) server_negotiation.cc:407] Connection header received
0808 14:40:34.107365 (+ 2661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:40:34.107370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:40:34.107439 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:40:34.107511 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:40:34.109454 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.110414 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.112526 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.112882 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.113006 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:34.116891 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:40:34.116897 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:40:34.116901 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:40:34.116938 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:40:34.119138 (+ 2200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:40:34.119919 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:40:34.119924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:40:34.119926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:40:34.120041 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:40:34.120581 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:40:34.120585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:40:34.120587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:40:34.120854 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:40:34.120942 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 14:40:34.121838 (+ 896us) server_negotiation.cc:299] Negotiation successful
0808 14:40:34.122005 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 14:40:34.124595 32015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.103740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:34645 (local address 127.19.144.66:57721)
0808 14:40:34.104488 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0808 14:40:34.104555 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 14:40:34.107038 (+ 2483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:40:34.108021 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:40:34.108032 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:40:34.108435 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:40:34.109241 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:40:34.109264 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.110555 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.110559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:40:34.112289 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:40:34.112305 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.113233 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.113238 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:40:34.113498 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:34.114368 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:40:34.114395 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:40:34.116379 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:40:34.119315 (+ 2936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:40:34.119323 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:40:34.119345 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:40:34.119772 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:40:34.120174 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:40:34.120177 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:40:34.120179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:40:34.120440 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:40:34.121041 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:40:34.121049 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:40:34.121439 (+ 390us) client_negotiation.cc:769] Sending connection context
0808 14:40:34.122795 (+ 1356us) client_negotiation.cc:240] Negotiation successful
0808 14:40:34.123272 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":188,"threads_started":1}
I20260808 14:40:34.126303 32012 heartbeater.cc:347] Connected to a master server at 127.19.144.126:34645
I20260808 14:40:34.126616 32012 heartbeater.cc:464] Registering TS with master...
I20260808 14:40:34.127300 32012 heartbeater.cc:511] Master 127.19.144.126:34645 requested a full tablet report, sending...
I20260808 14:40:34.128701 31654 ts_manager.cc:194] Registered new tserver with Master: 35c14622489a4cffaf32b9afc4403d97 (127.19.144.66:37973)
I20260808 14:40:34.129508 31654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:57721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:40:34.143919 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.144.84:55315/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34645
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:33579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:40:34 dist-test-slave-p3t5 krb5kdc[31162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200033, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
W20260808 14:40:34.327172 32017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:40:34.327620 32017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:40:34.327682 32017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:40:34.333900 32017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:40:34.334007 32017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:40:34.334096 32017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:40:34.342790 32017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:33579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.144.84:55315/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:40:34.344481 32017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:40:34.345898 32017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:40:34.358964 32023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:40:34.359054 32025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:40:34.359419 32022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:40:34.359769 32017 server_base.cc:1034] running on GCE node
I20260808 14:40:34.360270 32017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:40:34.361083 32017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:40:34.362278 32017 hybrid_clock.cc:648] HybridClock initialized: now 1786200034362254 us; error 38 us; skew 500 ppm
Aug 08 14:40:34 dist-test-slave-p3t5 krb5kdc[31162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200034, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:40:34.366261 32017 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:40:34.367471 32017 webserver.cc:460] Webserver started at http://127.19.144.67:33175/ using document root <none> and password file <none>
I20260808 14:40:34.368216 32017 fs_manager.cc:359] Metadata directory not provided
I20260808 14:40:34.368312 32017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:40:34.368624 32017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 14:40:34 dist-test-slave-p3t5 krb5kdc[31162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200034, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
I20260808 14:40:34.388492 32017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "eb5c13860a1d4f9880f607c30e810f88"
format_stamp: "Formatted at 2026-08-08 14:40:34 on dist-test-slave-p3t5"
server_key: "72bfb73c150239f238914e53e4001149"
server_key_iv: "2a741fcc9f084019e7d134ad5906e158"
server_key_version: "kuduclusterkey@0"
I20260808 14:40:34.389565 32017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "eb5c13860a1d4f9880f607c30e810f88"
format_stamp: "Formatted at 2026-08-08 14:40:34 on dist-test-slave-p3t5"
server_key: "72bfb73c150239f238914e53e4001149"
server_key_iv: "2a741fcc9f084019e7d134ad5906e158"
server_key_version: "kuduclusterkey@0"
I20260808 14:40:34.410825 32017 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.005s sys 0.000s
I20260808 14:40:34.435411 32033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:40:34.437518 32017 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.000s sys 0.001s
I20260808 14:40:34.437731 32017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "eb5c13860a1d4f9880f607c30e810f88"
format_stamp: "Formatted at 2026-08-08 14:40:34 on dist-test-slave-p3t5"
server_key: "72bfb73c150239f238914e53e4001149"
server_key_iv: "2a741fcc9f084019e7d134ad5906e158"
server_key_version: "kuduclusterkey@0"
I20260808 14:40:34.437985 32017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:40:34.490890 32017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:40:34.495352 32017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:40:34.495594 32017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:40:34.496415 32017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:40:34.497684 32017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:40:34.497766 32017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:40:34.497893 32017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:40:34.497933 32017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:40:34.511948 32017 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:36467
I20260808 14:40:34.513445 32017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:40:34.518060 32149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:36467 every 8 connection(s)
I20260808 14:40:34.523459 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 32017
I20260808 14:40:34.523582 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:40:34.523896 20033 external_mini_cluster.cc:1428] Setting key 58959d163f2813d812bb6479ce2a3b63
Aug 08 14:40:34 dist-test-slave-p3t5 krb5kdc[31162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200034, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:40:34.541247 31877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.522440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:45367 (local address 127.19.144.126:34645)
0808 14:40:34.522570 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 14:40:34.522574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:40:34.523034 (+ 460us) server_negotiation.cc:407] Connection header received
0808 14:40:34.527444 (+ 4410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:40:34.527448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:40:34.527506 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:40:34.527568 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:40:34.529395 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.530277 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.532106 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.532464 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.532551 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:34.535912 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:40:34.535916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:40:34.535919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:40:34.535948 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:40:34.538153 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:40:34.539024 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:40:34.539028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:40:34.539030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:40:34.539145 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:40:34.539709 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:40:34.539713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:40:34.539715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:40:34.540005 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:40:34.540114 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 14:40:34.540929 (+ 815us) server_negotiation.cc:299] Negotiation successful
0808 14:40:34.541078 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 14:40:34.542577 32152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.521174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:34645 (local address 127.19.144.67:45367)
0808 14:40:34.522859 (+ 1685us) negotiation.cc:107] Waiting for socket to connect
0808 14:40:34.522918 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 14:40:34.527117 (+ 4199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:40:34.528050 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:40:34.528060 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:40:34.528475 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:40:34.529217 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:40:34.529239 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.530457 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.530461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:40:34.531954 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:40:34.531966 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.532786 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.532791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:40:34.533014 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:34.533848 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:40:34.533871 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:40:34.535673 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:40:34.538314 (+ 2641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:40:34.538322 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:40:34.538349 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:40:34.538771 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:40:34.539275 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:40:34.539279 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:40:34.539281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:40:34.539520 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:40:34.540183 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:40:34.540191 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:40:34.540591 (+ 400us) client_negotiation.cc:769] Sending connection context
0808 14:40:34.540862 (+ 271us) client_negotiation.cc:240] Negotiation successful
0808 14:40:34.541311 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1452,"thread_start_us":206,"threads_started":1}
I20260808 14:40:34.543965 32150 heartbeater.cc:347] Connected to a master server at 127.19.144.126:34645
I20260808 14:40:34.544251 32150 heartbeater.cc:464] Registering TS with master...
I20260808 14:40:34.544997 32150 heartbeater.cc:511] Master 127.19.144.126:34645 requested a full tablet report, sending...
I20260808 14:40:34.546543 31654 ts_manager.cc:194] Registered new tserver with Master: eb5c13860a1d4f9880f607c30e810f88 (127.19.144.67:36467)
I20260808 14:40:34.547367 31654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:45367
I20260808 14:40:34.554893 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 14:40:34 dist-test-slave-p3t5 krb5kdc[31162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200033, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
I20260808 14:40:34.582741 31877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.563892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42606 (local address 127.19.144.126:34645)
0808 14:40:34.564037 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 14:40:34.564042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:40:34.568102 (+ 4060us) server_negotiation.cc:407] Connection header received
0808 14:40:34.568264 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:40:34.568268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:40:34.568330 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:40:34.568430 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:40:34.569433 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.570279 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.571650 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.571969 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.572089 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:34.578613 (+ 6524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:40:34.578618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:40:34.578621 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:40:34.578658 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:40:34.580502 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:40:34.581117 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:40:34.581121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:40:34.581123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:40:34.581227 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:40:34.581589 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:40:34.581593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:40:34.581595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:40:34.581888 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:40:34.582015 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 14:40:34.582413 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 14:40:34.582561 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:40:34.587764 31654 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42606:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:40:34.593225 31654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:40:34.634222 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.619793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35426 (local address 127.19.144.66:37973)
0808 14:40:34.621670 (+ 1877us) server_negotiation.cc:206] Beginning negotiation
0808 14:40:34.621686 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:40:34.622075 (+ 389us) server_negotiation.cc:407] Connection header received
0808 14:40:34.622162 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:40:34.622168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:40:34.623169 (+ 1001us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:40:34.623553 (+ 384us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:40:34.624037 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.625949 (+ 1912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.630341 (+ 4392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.632362 (+ 2021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.632474 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:34.632651 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 14:40:34.633945 (+ 1294us) server_negotiation.cc:299] Negotiation successful
0808 14:40:34.634069 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1743,"thread_start_us":145,"threads_started":1}
I20260808 14:40:34.633400 32167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.619156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:37973 (local address 127.0.0.1:35426)
0808 14:40:34.621832 (+ 2676us) negotiation.cc:107] Waiting for socket to connect
0808 14:40:34.621847 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:40:34.621968 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:40:34.623723 (+ 1755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:40:34.623726 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:40:34.623744 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:40:34.623863 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:40:34.623870 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.626099 (+ 2229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.626103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:40:34.630172 (+ 4069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:40:34.630187 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.632720 (+ 2533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.632724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:40:34.632867 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:34.632942 (+ 75us) client_negotiation.cc:769] Sending connection context
0808 14:40:34.633062 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 14:40:34.633229 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2594,"thread_start_us":107,"threads_started":1}
I20260808 14:40:34.638087 31946 tablet_service.cc:1467] Processing CreateTablet for tablet 81dabc560918496192b611223e660dd8 (DEFAULT_TABLE table=test-table [id=2a55c612bcb74092acef159e94f74b50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:40:34.639521 31946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81dabc560918496192b611223e660dd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:40:34.639588 32165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.618378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:43111 (local address 127.0.0.1:38630)
0808 14:40:34.621792 (+ 3414us) negotiation.cc:107] Waiting for socket to connect
0808 14:40:34.621810 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:40:34.621941 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:40:34.622741 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:40:34.622749 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:40:34.622778 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:40:34.622958 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:40:34.622967 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.636532 (+ 13565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.636535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:40:34.637603 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:40:34.637653 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.639116 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.639120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:40:34.639225 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:34.639257 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 14:40:34.639333 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 14:40:34.639425 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3333,"thread_start_us":104,"threads_started":1}
I20260808 14:40:34.640156 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.619280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38630 (local address 127.19.144.65:43111)
0808 14:40:34.621441 (+ 2161us) server_negotiation.cc:206] Beginning negotiation
0808 14:40:34.621450 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:40:34.622061 (+ 611us) server_negotiation.cc:407] Connection header received
0808 14:40:34.622162 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:40:34.622168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:40:34.622422 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:40:34.622601 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:40:34.626471 (+ 3870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.631812 (+ 5341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.637812 (+ 6000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.638750 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.638868 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:34.639048 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 14:40:34.639860 (+ 812us) server_negotiation.cc:299] Negotiation successful
0808 14:40:34.640005 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2030,"thread_start_us":88,"threads_started":1}
I20260808 14:40:34.643558 31805 tablet_service.cc:1467] Processing CreateTablet for tablet 81dabc560918496192b611223e660dd8 (DEFAULT_TABLE table=test-table [id=2a55c612bcb74092acef159e94f74b50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:40:34.644901 31805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81dabc560918496192b611223e660dd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:40:34.655588 32171 tablet_bootstrap.cc:492] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97: Bootstrap starting.
I20260808 14:40:34.660593 32168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.616651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:36467 (local address 127.0.0.1:51422)
0808 14:40:34.619640 (+ 2989us) negotiation.cc:107] Waiting for socket to connect
0808 14:40:34.619708 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 14:40:34.619862 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:40:34.631600 (+ 11738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:40:34.631604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:40:34.631634 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:40:34.633555 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:40:34.633564 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.635808 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.635812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:40:34.658468 (+ 22656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:40:34.658489 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.660107 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.660112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:40:34.660212 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:34.660241 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 14:40:34.660316 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 14:40:34.660424 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2784,"thread_start_us":151,"threads_started":1}
I20260808 14:40:34.661206 32166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.617669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51422 (local address 127.19.144.67:36467)
0808 14:40:34.628235 (+ 10566us) server_negotiation.cc:206] Beginning negotiation
0808 14:40:34.628250 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:40:34.628280 (+ 30us) server_negotiation.cc:407] Connection header received
0808 14:40:34.628370 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:40:34.628378 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:40:34.628658 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:40:34.628841 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:40:34.633861 (+ 5020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.635548 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.658732 (+ 23184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.659715 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.659844 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:34.660022 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 14:40:34.660931 (+ 909us) server_negotiation.cc:299] Negotiation successful
0808 14:40:34.661055 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10422,"thread_start_us":175,"threads_started":1}
I20260808 14:40:34.664759 32081 tablet_service.cc:1467] Processing CreateTablet for tablet 81dabc560918496192b611223e660dd8 (DEFAULT_TABLE table=test-table [id=2a55c612bcb74092acef159e94f74b50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:40:34.665767 32172 tablet_bootstrap.cc:492] T 81dabc560918496192b611223e660dd8 P 42b3be04b51c4114ac973d8136b70a74: Bootstrap starting.
I20260808 14:40:34.666289 32081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81dabc560918496192b611223e660dd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:40:34.668812 32172 tablet_bootstrap.cc:654] T 81dabc560918496192b611223e660dd8 P 42b3be04b51c4114ac973d8136b70a74: Neither blocks nor log segments found. Creating new log.
I20260808 14:40:34.669911 32172 log.cc:824] T 81dabc560918496192b611223e660dd8 P 42b3be04b51c4114ac973d8136b70a74: Log is configured to *not* fsync() on all Append() calls
I20260808 14:40:34.676626 32171 tablet_bootstrap.cc:654] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97: Neither blocks nor log segments found. Creating new log.
I20260808 14:40:34.677533 32171 log.cc:824] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97: Log is configured to *not* fsync() on all Append() calls
I20260808 14:40:34.679898 32175 tablet_bootstrap.cc:492] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88: Bootstrap starting.
I20260808 14:40:34.682760 32172 tablet_bootstrap.cc:492] T 81dabc560918496192b611223e660dd8 P 42b3be04b51c4114ac973d8136b70a74: No bootstrap required, opened a new log
I20260808 14:40:34.683027 32172 ts_tablet_manager.cc:1397] T 81dabc560918496192b611223e660dd8 P 42b3be04b51c4114ac973d8136b70a74: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.000s
I20260808 14:40:34.685243 32175 tablet_bootstrap.cc:654] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88: Neither blocks nor log segments found. Creating new log.
I20260808 14:40:34.686441 32175 log.cc:824] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88: Log is configured to *not* fsync() on all Append() calls
I20260808 14:40:34.688180 32171 tablet_bootstrap.cc:492] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97: No bootstrap required, opened a new log
I20260808 14:40:34.688150 32172 raft_consensus.cc:348] T 81dabc560918496192b611223e660dd8 P 42b3be04b51c4114ac973d8136b70a74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c13860a1d4f9880f607c30e810f88" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36467 } } peers { permanent_uuid: "42b3be04b51c4114ac973d8136b70a74" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43111 } } peers { permanent_uuid: "35c14622489a4cffaf32b9afc4403d97" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 37973 } }
I20260808 14:40:34.688446 32171 ts_tablet_manager.cc:1397] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.003s
I20260808 14:40:34.688503 32172 raft_consensus.cc:374] T 81dabc560918496192b611223e660dd8 P 42b3be04b51c4114ac973d8136b70a74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:40:34.688565 32172 raft_consensus.cc:729] T 81dabc560918496192b611223e660dd8 P 42b3be04b51c4114ac973d8136b70a74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42b3be04b51c4114ac973d8136b70a74, State: Initialized, Role: FOLLOWER
I20260808 14:40:34.689079 32172 consensus_queue.cc:260] T 81dabc560918496192b611223e660dd8 P 42b3be04b51c4114ac973d8136b70a74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c13860a1d4f9880f607c30e810f88" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36467 } } peers { permanent_uuid: "42b3be04b51c4114ac973d8136b70a74" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43111 } } peers { permanent_uuid: "35c14622489a4cffaf32b9afc4403d97" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 37973 } }
I20260808 14:40:34.689875 31874 heartbeater.cc:503] Master 127.19.144.126:34645 was elected leader, sending a full tablet report...
I20260808 14:40:34.690301 32172 ts_tablet_manager.cc:1428] T 81dabc560918496192b611223e660dd8 P 42b3be04b51c4114ac973d8136b70a74: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260808 14:40:34.693318 32171 raft_consensus.cc:348] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c13860a1d4f9880f607c30e810f88" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36467 } } peers { permanent_uuid: "42b3be04b51c4114ac973d8136b70a74" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43111 } } peers { permanent_uuid: "35c14622489a4cffaf32b9afc4403d97" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 37973 } }
I20260808 14:40:34.697719 32171 raft_consensus.cc:374] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:40:34.697889 32171 raft_consensus.cc:729] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c14622489a4cffaf32b9afc4403d97, State: Initialized, Role: FOLLOWER
I20260808 14:40:34.698552 32171 consensus_queue.cc:260] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c13860a1d4f9880f607c30e810f88" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36467 } } peers { permanent_uuid: "42b3be04b51c4114ac973d8136b70a74" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43111 } } peers { permanent_uuid: "35c14622489a4cffaf32b9afc4403d97" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 37973 } }
I20260808 14:40:34.700500 32175 tablet_bootstrap.cc:492] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88: No bootstrap required, opened a new log
I20260808 14:40:34.700764 32175 ts_tablet_manager.cc:1397] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.004s
I20260808 14:40:34.702190 32171 ts_tablet_manager.cc:1428] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97: Time spent starting tablet: real 0.014s user 0.004s sys 0.004s
I20260808 14:40:34.702617 32012 heartbeater.cc:503] Master 127.19.144.126:34645 was elected leader, sending a full tablet report...
I20260808 14:40:34.709699 32175 raft_consensus.cc:348] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c13860a1d4f9880f607c30e810f88" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36467 } } peers { permanent_uuid: "42b3be04b51c4114ac973d8136b70a74" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43111 } } peers { permanent_uuid: "35c14622489a4cffaf32b9afc4403d97" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 37973 } }
I20260808 14:40:34.710178 32175 raft_consensus.cc:374] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:40:34.710314 32175 raft_consensus.cc:729] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5c13860a1d4f9880f607c30e810f88, State: Initialized, Role: FOLLOWER
I20260808 14:40:34.710985 32175 consensus_queue.cc:260] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c13860a1d4f9880f607c30e810f88" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36467 } } peers { permanent_uuid: "42b3be04b51c4114ac973d8136b70a74" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43111 } } peers { permanent_uuid: "35c14622489a4cffaf32b9afc4403d97" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 37973 } }
I20260808 14:40:34.712045 32150 heartbeater.cc:503] Master 127.19.144.126:34645 was elected leader, sending a full tablet report...
I20260808 14:40:34.712363 32175 ts_tablet_manager.cc:1428] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
W20260808 14:40:34.734685 31875 tablet.cc:2369] T 81dabc560918496192b611223e660dd8 P 42b3be04b51c4114ac973d8136b70a74: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:40:34.768970 32151 tablet.cc:2369] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:40:34.807542 32179 raft_consensus.cc:482] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:40:34.807832 32179 raft_consensus.cc:504] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c13860a1d4f9880f607c30e810f88" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36467 } } peers { permanent_uuid: "42b3be04b51c4114ac973d8136b70a74" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43111 } } peers { permanent_uuid: "35c14622489a4cffaf32b9afc4403d97" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 37973 } }
I20260808 14:40:34.809458 32179 leader_election.cc:288] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42b3be04b51c4114ac973d8136b70a74 (127.19.144.65:43111), 35c14622489a4cffaf32b9afc4403d97 (127.19.144.66:37973)
I20260808 14:40:34.815310 32152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.809737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:43111 (local address 127.19.144.67:44029)
0808 14:40:34.810079 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 14:40:34.810095 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:40:34.810207 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:40:34.810821 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:40:34.810824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:40:34.810844 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:40:34.810976 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:40:34.810983 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.812076 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.812080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:40:34.813364 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:40:34.813376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.814763 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.814768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:40:34.814891 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:34.814976 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 14:40:34.815058 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 14:40:34.815145 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":16}
I20260808 14:40:34.815426 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.810294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:57521 (local address 127.19.144.66:37973)
0808 14:40:34.810429 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 14:40:34.810434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:40:34.810818 (+ 384us) server_negotiation.cc:407] Connection header received
0808 14:40:34.811020 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:40:34.811023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:40:34.811074 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:40:34.811123 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:40:34.811562 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.812351 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.813926 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.814571 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.814666 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:34.814715 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 14:40:34.815172 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 14:40:34.815263 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 14:40:34.816169 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.810420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:44029 (local address 127.19.144.65:43111)
0808 14:40:34.810550 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 14:40:34.810557 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:40:34.810576 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:40:34.810634 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:40:34.810636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:40:34.810679 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:40:34.810796 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:40:34.811110 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.811913 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.813550 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.814242 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.814374 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:34.814433 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 14:40:34.815942 (+ 1509us) server_negotiation.cc:299] Negotiation successful
0808 14:40:34.816041 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 14:40:34.816884 31825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81dabc560918496192b611223e660dd8" candidate_uuid: "eb5c13860a1d4f9880f607c30e810f88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42b3be04b51c4114ac973d8136b70a74" is_pre_election: true
I20260808 14:40:34.817030 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.809847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:37973 (local address 127.19.144.67:57521)
0808 14:40:34.810713 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0808 14:40:34.810727 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:40:34.810897 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:40:34.811262 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:40:34.811265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:40:34.811283 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:40:34.811416 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:40:34.811423 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.812516 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.812519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:40:34.813774 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:40:34.813785 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.814826 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.814830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:40:34.814930 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:34.815043 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 14:40:34.816785 (+ 1742us) client_negotiation.cc:240] Negotiation successful
0808 14:40:34.816876 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"mutex_wait_us":76,"thread_start_us":140,"threads_started":1}
I20260808 14:40:34.817262 31825 raft_consensus.cc:2454] T 81dabc560918496192b611223e660dd8 P 42b3be04b51c4114ac973d8136b70a74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb5c13860a1d4f9880f607c30e810f88 in term 0.
I20260808 14:40:34.817684 31966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81dabc560918496192b611223e660dd8" candidate_uuid: "eb5c13860a1d4f9880f607c30e810f88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c14622489a4cffaf32b9afc4403d97" is_pre_election: true
I20260808 14:40:34.818120 32036 leader_election.cc:302] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42b3be04b51c4114ac973d8136b70a74, eb5c13860a1d4f9880f607c30e810f88; no voters:
I20260808 14:40:34.818306 31966 raft_consensus.cc:2454] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb5c13860a1d4f9880f607c30e810f88 in term 0.
I20260808 14:40:34.818611 32179 raft_consensus.cc:2781] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:40:34.818710 32179 raft_consensus.cc:482] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:40:34.818753 32179 raft_consensus.cc:3037] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:40:34.820592 32179 raft_consensus.cc:504] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c13860a1d4f9880f607c30e810f88" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36467 } } peers { permanent_uuid: "42b3be04b51c4114ac973d8136b70a74" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43111 } } peers { permanent_uuid: "35c14622489a4cffaf32b9afc4403d97" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 37973 } }
I20260808 14:40:34.821055 32179 leader_election.cc:288] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [CANDIDATE]: Term 1 election: Requested vote from peers 42b3be04b51c4114ac973d8136b70a74 (127.19.144.65:43111), 35c14622489a4cffaf32b9afc4403d97 (127.19.144.66:37973)
I20260808 14:40:34.821445 31825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81dabc560918496192b611223e660dd8" candidate_uuid: "eb5c13860a1d4f9880f607c30e810f88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42b3be04b51c4114ac973d8136b70a74"
I20260808 14:40:34.821547 31966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81dabc560918496192b611223e660dd8" candidate_uuid: "eb5c13860a1d4f9880f607c30e810f88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c14622489a4cffaf32b9afc4403d97"
I20260808 14:40:34.821583 31825 raft_consensus.cc:3037] T 81dabc560918496192b611223e660dd8 P 42b3be04b51c4114ac973d8136b70a74 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:40:34.821703 31966 raft_consensus.cc:3037] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:40:34.822956 31825 raft_consensus.cc:2454] T 81dabc560918496192b611223e660dd8 P 42b3be04b51c4114ac973d8136b70a74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c13860a1d4f9880f607c30e810f88 in term 1.
I20260808 14:40:34.823316 32036 leader_election.cc:302] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42b3be04b51c4114ac973d8136b70a74, eb5c13860a1d4f9880f607c30e810f88; no voters:
I20260808 14:40:34.823333 31966 raft_consensus.cc:2454] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5c13860a1d4f9880f607c30e810f88 in term 1.
I20260808 14:40:34.823555 32179 raft_consensus.cc:2781] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:40:34.823817 32179 raft_consensus.cc:686] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [term 1 LEADER]: Becoming Leader. State: Replica: eb5c13860a1d4f9880f607c30e810f88, State: Running, Role: LEADER
I20260808 14:40:34.824147 32179 consensus_queue.cc:237] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c13860a1d4f9880f607c30e810f88" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36467 } } peers { permanent_uuid: "42b3be04b51c4114ac973d8136b70a74" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43111 } } peers { permanent_uuid: "35c14622489a4cffaf32b9afc4403d97" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 37973 } }
I20260808 14:40:34.827736 31653 catalog_manager.cc:5495] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb5c13860a1d4f9880f607c30e810f88 (127.19.144.67). New cstate: current_term: 1 leader_uuid: "eb5c13860a1d4f9880f607c30e810f88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5c13860a1d4f9880f607c30e810f88" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42b3be04b51c4114ac973d8136b70a74" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35c14622489a4cffaf32b9afc4403d97" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 37973 } health_report { overall_health: UNKNOWN } } }
W20260808 14:40:34.852473 32013 tablet.cc:2369] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:40:34.872110 32166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.868279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51424 (local address 127.19.144.67:36467)
0808 14:40:34.868428 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 14:40:34.868432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:40:34.868450 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:40:34.868574 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:40:34.868577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:40:34.868634 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:40:34.868755 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:40:34.869029 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.869552 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.870512 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:40:34.870743 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:40:34.870853 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:40:34.871201 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:40:34.871669 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:40:34.871837 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 14:40:34.871871 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 14:40:34.871934 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 14:40:34.879480 31825 raft_consensus.cc:1261] T 81dabc560918496192b611223e660dd8 P 42b3be04b51c4114ac973d8136b70a74 [term 1 FOLLOWER]: Refusing update from remote peer eb5c13860a1d4f9880f607c30e810f88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:40:34.879523 31966 raft_consensus.cc:1261] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97 [term 1 FOLLOWER]: Refusing update from remote peer eb5c13860a1d4f9880f607c30e810f88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:40:34.880401 32181 consensus_queue.cc:1035] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42b3be04b51c4114ac973d8136b70a74" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 43111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:40:34.880656 32179 consensus_queue.cc:1035] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35c14622489a4cffaf32b9afc4403d97" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 37973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:40:34.888975 32192 mvcc.cc:205] Tried to move back new op lower bound from 7316275342857728000 to 7316275342642884608. Current Snapshot: MvccSnapshot[applied={T|T < 7316275342857728000}]
I20260808 14:40:34.892781 32194 mvcc.cc:205] Tried to move back new op lower bound from 7316275342857728000 to 7316275342642884608. Current Snapshot: MvccSnapshot[applied={T|T < 7316275342857728000}]
I20260808 14:40:34.898577 31653 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:40:34.898769 31653 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:40:34.901710 31653 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0130f457890a463c880d078156bbaa38: Sending DeleteTablet for 3 replicas of tablet 81dabc560918496192b611223e660dd8
I20260808 14:40:34.902551 31946 tablet_service.cc:1514] Processing DeleteTablet for tablet 81dabc560918496192b611223e660dd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:40:34 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:35426
I20260808 14:40:34.902647 31805 tablet_service.cc:1514] Processing DeleteTablet for tablet 81dabc560918496192b611223e660dd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:40:34 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:38630
I20260808 14:40:34.903333 32081 tablet_service.cc:1514] Processing DeleteTablet for tablet 81dabc560918496192b611223e660dd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:40:34 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:51422
I20260808 14:40:34.903861 32203 tablet_replica.cc:330] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88: stopping tablet replica
I20260808 14:40:34.904242 32203 raft_consensus.cc:2229] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:40:34.904316 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 31738
I20260808 14:40:34.904671 32203 raft_consensus.cc:2258] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:40:34.905848 32201 tablet_replica.cc:330] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97: stopping tablet replica
I20260808 14:40:34.906150 32201 raft_consensus.cc:2229] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:40:34.906495 32201 raft_consensus.cc:2258] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:40:34.911273 32203 ts_tablet_manager.cc:1905] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:40:34.914589 32203 ts_tablet_manager.cc:1918] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:40:34.914801 32203 log.cc:1196] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/81dabc560918496192b611223e660dd8
I20260808 14:40:34.914919 32201 ts_tablet_manager.cc:1905] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:40:34.915350 32203 ts_tablet_manager.cc:1939] T 81dabc560918496192b611223e660dd8 P eb5c13860a1d4f9880f607c30e810f88: Deleting consensus metadata
I20260808 14:40:34.916955 31640 catalog_manager.cc:4841] TS eb5c13860a1d4f9880f607c30e810f88 (127.19.144.67:36467): tablet 81dabc560918496192b611223e660dd8 (table test-table [id=2a55c612bcb74092acef159e94f74b50]) successfully deleted
I20260808 14:40:34.918128 32201 ts_tablet_manager.cc:1918] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:40:34.918334 32201 log.cc:1196] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/81dabc560918496192b611223e660dd8
I20260808 14:40:34.918773 32201 ts_tablet_manager.cc:1939] T 81dabc560918496192b611223e660dd8 P 35c14622489a4cffaf32b9afc4403d97: Deleting consensus metadata
I20260808 14:40:34.920342 31639 catalog_manager.cc:4841] TS 35c14622489a4cffaf32b9afc4403d97 (127.19.144.66:37973): tablet 81dabc560918496192b611223e660dd8 (table test-table [id=2a55c612bcb74092acef159e94f74b50]) successfully deleted
W20260808 14:40:34.921757 31640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.144.65:43111 (error 108)
I20260808 14:40:34.922766 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 31880
I20260808 14:40:34.923556 32168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:40:34.922568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:43111 (local address 127.0.0.1:38634)
0808 14:40:34.923389 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0808 14:40:34.923474 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":705}
W20260808 14:40:34.926472 31640 catalog_manager.cc:4584] TS 42b3be04b51c4114ac973d8136b70a74 (127.19.144.65:43111): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 81dabc560918496192b611223e660dd8: Network error: Client connection negotiation failed: client connection to 127.19.144.65:43111: connect: Connection refused (error 111)
I20260808 14:40:34.933027 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 32017
I20260808 14:40:34.942097 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 31622
I20260808 14:40:34.951660 20033 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 14:40:35.297117 20033 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 14:40:35.297268 20033 mini_ranger.cc:66] Stopping Ranger...
I20260808 14:40:35.642990 20033 mini_ranger.cc:68] Stopped Ranger
2026-08-08 14:40:35.643 UTC [31194] LOG: received smart shutdown request
2026-08-08 14:40:35.647 UTC [31194] LOG: background worker "logical replication launcher" (PID 31203) exited with exit code 1
2026-08-08 14:40:35.648 UTC [31198] LOG: shutting down
2026-08-08 14:40:35.837 UTC [31194] LOG: database system is shut down
2026-08-08T14:40:35Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49081 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:40:36 dist-test-slave-p3t5 krb5kdc[32213](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:40:36 dist-test-slave-p3t5 krb5kdc[32213](info): set up 2 sockets
Aug 08 14:40:36 dist-test-slave-p3t5 krb5kdc[32213](info): commencing operation
krb5kdc: starting...
W20260808 14:40:37.041702 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:40:37 dist-test-slave-p3t5 krb5kdc[32213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:40:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:40:37Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.144.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.144.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.144.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/rangeradmin_127.19.144.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.144.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/rangeradmin_127.19.144.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.144.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.144.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.144.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/rangerlookup_127.19.144.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.144.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/rangerlookup_127.19.144.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.144.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.144.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.144.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/HTTP_127.19.144.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.144.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/HTTP_127.19.144.84@KRBTEST.COM.keytab.
I20260808 14:40:37.236827 20033 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskCQkDbp/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/postgres -l logfile start
2026-08-08 14:40:39.876 UTC [32243] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 14:40:39.876 UTC [32243] LOG: listening on IPv4 address "127.19.144.84", port 55725
2026-08-08 14:40:39.880 UTC [32243] LOG: listening on Unix socket "/tmp/.s.PGSQL.55725"
2026-08-08 14:40:39.903 UTC [32246] LOG: database system was shut down at 2026-08-08 14:40:38 UTC
2026-08-08 14:40:39.907 UTC [32243] LOG: database system is ready to accept connections
I20260808 14:40:40.903141 20033 mini_postgres.cc:96] Postgres bound to 55725
2026-08-08 14:40:40.911 UTC [32255] FATAL: database "slave" does not exist
127.19.144.84:55725 - accepting connections
I20260808 14:40:40.911702 20033 mini_ranger.cc:161] Starting Ranger...
I20260808 14:40:40.928468 20033 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 14:40:41.447209 20033 mini_ranger.cc:87] Created ranger Postgres database
I20260808 14:40:41.447304 20033 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-admin
2026-08-08 14:40:41,742 [I] DB FLAVOR :POSTGRES
2026-08-08 14:40:41,742 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 14:40:41,742 [I] Checking connection..
2026-08-08 14:40:41,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 14:40:41,997 [I] Checking connection passed.
2026-08-08 14:40:41,998 [I] --------- Verifying version history table ---------
2026-08-08 14:40:41,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 14:40:42,273 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 14:40:42,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 14:40:42,564 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 14:40:42,565 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 14:40:42,565 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 14:40:42.838 UTC [32320] WARNING: there is no transaction in progress
2026-08-08 14:40:42,846 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 14:40:42,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 14:40:43,126 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 14:40:43,127 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 14:40:43,127 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 14:40:43,386 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 14:40:43,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-p3t5.c.gcp-upstream.internal','N') ;"
2026-08-08 14:40:43,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 14:40:44,207 [I] Table x_portal_user does not exist in database ranger
2026-08-08 14:40:44,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 14:40:44,478 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 14:40:44,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 14:40:44,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 14:40:44,993 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 14:40:44,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 14:40:46.063 UTC [32431] WARNING: there is no transaction in progress
2026-08-08 14:40:46.080 UTC [32431] WARNING: there is no transaction in progress
2026-08-08 14:40:46.097 UTC [32431] WARNING: there is no transaction in progress
2026-08-08 14:40:46.113 UTC [32431] WARNING: there is no transaction in progress
2026-08-08 14:40:46.133 UTC [32431] WARNING: there is no transaction in progress
2026-08-08 14:40:46.153 UTC [32431] WARNING: there is no transaction in progress
2026-08-08 14:40:46.214 UTC [32431] WARNING: there is no transaction in progress
2026-08-08 14:40:46.223 UTC [32431] WARNING: there is no transaction in progress
2026-08-08 14:40:46.239 UTC [32431] WARNING: there is no transaction in progress
2026-08-08 14:40:46.250 UTC [32431] WARNING: there is no transaction in progress
2026-08-08 14:40:46.261 UTC [32431] WARNING: there is no transaction in progress
2026-08-08 14:40:46.276 UTC [32431] WARNING: there is no transaction in progress
2026-08-08 14:40:46.286 UTC [32431] WARNING: there is no transaction in progress
2026-08-08 14:40:46.294 UTC [32431] WARNING: there is no transaction in progress
2026-08-08 14:40:46.301 UTC [32431] WARNING: there is no transaction in progress
2026-08-08 14:40:46.309 UTC [32431] WARNING: there is no transaction in progress
2026-08-08 14:40:47,044 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 14:40:47,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 14:40:47,289 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 14:40:47,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 14:40:47,544 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 14:40:47,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 14:40:47,829 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 14:40:47,830 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 14:40:48,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 14:40:48,366 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-p3t5.c.gcp-upstream.internal';"
2026-08-08 14:40:48,607 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 14:40:48,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 14:40:48,888 [I] DB_PATCHES have already been applied
I20260808 14:40:48.895113 20033 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-admin:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskCQkDbp/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 14:40:48.895193 20033 mini_ranger.cc:193] Using host: 127.19.144.84
I20260808 14:40:48.899860 20033 mini_ranger.cc:227] Ranger admin URL: http://127.19.144.84:49755
Aug 08, 2026 2:40:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:40:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:40:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 2:40:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 2:40:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 2:40:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 2:40:49 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 14:40:50.027206 20033 mini_ranger.cc:160] Time spent starting Ranger: real 9.116s user 0.000s sys 0.013s
Aug 08, 2026 2:41:11 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 2:41:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 2:41:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 2:41:11 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 2:41:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 2:41:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 2:41:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 2:41:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 2:41:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 2:41:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 2:41:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 2:41:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 2:41:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 2:41:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 2:41:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 2:41:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 2:41:11 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 2:41:12 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 14:41:12.998587 20033 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.144.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.144.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.144.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/rangerkms_127.19.144.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.144.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/rangerkms_127.19.144.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.144.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.144.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.144.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/HTTP_127.19.144.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.144.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/HTTP_127.19.144.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 14:41:13 dist-test-slave-p3t5 krb5kdc[32213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200073, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 14:41:13.094467 20033 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 14:41:13.105582 20033 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 14:41:14.076383 20033 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 14:41:14.076472 20033 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-kms
I20260808 14:41:14.076486 20033 mini_ranger_kms.cc:227] Using postgres at 127.19.144.84:55725
2026-08-08 14:41:14,259 [I] DB FLAVOR :POSTGRES
2026-08-08 14:41:14,260 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 14:41:14,260 [I] Checking connection
2026-08-08 14:41:14,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 14:41:14,501 [I] connection success
2026-08-08 14:41:14,502 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 14:41:14,502 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 14:41:14,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 14:41:14,745 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 14:41:14,746 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 14:41:14,746 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 14:41:14,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.144.84:55725/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 14:41:15,022 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 14:41:15.510427 20033 mini_ranger_kms.cc:310] Created kms service
I20260808 14:41:15.701105 20033 mini_ranger_kms.cc:326] Created kudu user
I20260808 14:41:15.745425 20033 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 14:41:15 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200073, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
I20260808 14:41:16.214103 20033 mini_ranger_kms.cc:384] Added ranger policy
I20260808 14:41:16.214282 20033 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-kms:/tmp/dist-test-taskCQkDbp/build/debug/bin/postgresql.jar:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskCQkDbp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 14:41:16.214305 20033 mini_ranger_kms.cc:240] Using host: 127.19.144.84
I20260808 14:41:16.220049 20033 mini_ranger_kms.cc:276] Ranger KMS PID: 32619
I20260808 14:41:16.220153 20033 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.144.84:49201
Aug 08, 2026 2:41:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:41:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:41:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 2:41:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 2:41:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 2:41:17 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 14:41:18 dist-test-slave-p3t5 krb5kdc[32213](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786200078, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.144.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:18.574880 20033 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.480s user 0.015s sys 0.152s
I20260808 14:41:18.575023 20033 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 14:41:18.575120 20033 mini_ranger_kms.cc:401] 127.19.144.84:49201/kms/v1/keys
Aug 08 14:41:18 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200078, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.144.84@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
Aug 08 14:41:19 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200078, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.144.84@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
Aug 08 14:41:21 dist-test-slave-p3t5 krb5kdc[32213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:41:21.584954 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:35155
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:35155
--ranger_config_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.144.84:49201/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:41:21.703402 32670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:21.703696 32670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:21.703737 32670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:21.707867 32670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:41:21.707942 32670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:21.707962 32670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:21.707978 32670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:41:21.707993 32670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:41:21.713842 32670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.144.84:49201/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:35155
--ranger_config_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:35155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:21.714942 32670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:21.715956 32670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:21.725708 32675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:21.725764 32670 server_base.cc:1034] running on GCE node
W20260808 14:41:21.725673 32678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:21.725699 32676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:21.726689 32670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:21.727906 32670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:21.729143 32670 hybrid_clock.cc:648] HybridClock initialized: now 1786200081729129 us; error 28 us; skew 500 ppm
Aug 08 14:41:21 dist-test-slave-p3t5 krb5kdc[32213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200081, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:21.736815 32670 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:41:21.737926 32670 webserver.cc:460] Webserver started at http://127.19.144.126:44475/ using document root <none> and password file <none>
I20260808 14:41:21.738494 32670 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:21.738552 32670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:21.738734 32670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 14:41:21 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200081, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
I20260808 14:41:21.798846 32670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "01554fc3f1444290abe4fef578e78c2e"
format_stamp: "Formatted at 2026-08-08 14:41:21 on dist-test-slave-p3t5"
server_key: "8db7c820d701c2a7aba7fab28c289f7a"
server_key_iv: "c98d9bbd9f030c9946c2773022ea5c37"
server_key_version: "kuduclusterkey@0"
I20260808 14:41:21.799577 32670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "01554fc3f1444290abe4fef578e78c2e"
format_stamp: "Formatted at 2026-08-08 14:41:21 on dist-test-slave-p3t5"
server_key: "8db7c820d701c2a7aba7fab28c289f7a"
server_key_iv: "c98d9bbd9f030c9946c2773022ea5c37"
server_key_version: "kuduclusterkey@0"
I20260808 14:41:21.804140 32670 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:41:21.819268 32686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:21.820837 32670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 14:41:21.820967 32670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "01554fc3f1444290abe4fef578e78c2e"
format_stamp: "Formatted at 2026-08-08 14:41:21 on dist-test-slave-p3t5"
server_key: "8db7c820d701c2a7aba7fab28c289f7a"
server_key_iv: "c98d9bbd9f030c9946c2773022ea5c37"
server_key_version: "kuduclusterkey@0"
I20260808 14:41:21.821148 32670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:21.837786 32670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:21.841193 32670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:21.841418 32670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:21.850272 32670 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:35155
I20260808 14:41:21.850274 32738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:35155 every 8 connection(s)
I20260808 14:41:21.851359 32670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:41:21.854425 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 32670
I20260808 14:41:21.854559 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:41:21.854611 32739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:21.854899 20033 external_mini_cluster.cc:1428] Setting key a79de20afd2be88d818dd098a602b550
I20260808 14:41:21.862345 32739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e: Bootstrap starting.
I20260808 14:41:21.865257 32739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:21.866300 32739 log.cc:824] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:41:21 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:21.869278 32739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e: No bootstrap required, opened a new log
I20260808 14:41:21.873569 32739 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01554fc3f1444290abe4fef578e78c2e" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35155 } }
I20260808 14:41:21.873821 32742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:21.856592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34220 (local address 127.19.144.126:35155)
0808 14:41:21.857104 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:21.857124 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:21.857172 (+ 48us) server_negotiation.cc:407] Connection header received
0808 14:41:21.858225 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:21.858254 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:21.858754 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:21.859147 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:21.860104 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:21.861678 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:21.862996 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:21.863703 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:21.863878 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:21.867203 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:21.867210 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:21.867226 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:21.867256 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:21.870173 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:21.870762 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:21.870768 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:21.870775 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:21.870950 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:21.871308 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:21.871311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:21.871313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:21.871875 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:21.872083 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:21.872418 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 14:41:21.872740 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":146,"threads_started":1}
I20260808 14:41:21.873915 32739 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:21.873965 32739 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01554fc3f1444290abe4fef578e78c2e, State: Initialized, Role: FOLLOWER
I20260808 14:41:21.874413 32739 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01554fc3f1444290abe4fef578e78c2e" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35155 } }
I20260808 14:41:21.874589 32739 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:41:21.874679 32739 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:41:21.874805 32739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:21.876075 32739 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01554fc3f1444290abe4fef578e78c2e" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35155 } }
I20260808 14:41:21.876471 32739 leader_election.cc:302] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01554fc3f1444290abe4fef578e78c2e; no voters:
I20260808 14:41:21.876870 32739 leader_election.cc:288] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:41:21.877012 32744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:41:21.877240 32744 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 1 LEADER]: Becoming Leader. State: Replica: 01554fc3f1444290abe4fef578e78c2e, State: Running, Role: LEADER
I20260808 14:41:21.877715 32744 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01554fc3f1444290abe4fef578e78c2e" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35155 } }
I20260808 14:41:21.878360 32739 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:41:21.879592 32746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01554fc3f1444290abe4fef578e78c2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01554fc3f1444290abe4fef578e78c2e" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35155 } } }
I20260808 14:41:21.879812 32746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [sys.catalog]: This master's current role is: LEADER
I20260808 14:41:21.880112 32746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01554fc3f1444290abe4fef578e78c2e. Latest consensus state: current_term: 1 leader_uuid: "01554fc3f1444290abe4fef578e78c2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01554fc3f1444290abe4fef578e78c2e" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35155 } } }
I20260808 14:41:21.880254 32746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [sys.catalog]: This master's current role is: LEADER
I20260808 14:41:21.880802 32739 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 14:41:21.885982 32755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:41:21.890635 32755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:41:21.897648 32755 catalog_manager.cc:1347] Generated new cluster ID: 654c797ea63e47b685c61025436a974b
I20260808 14:41:21.897729 32755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:41:21.915408 32755 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:41:21.916939 32755 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:41:21.924624 32755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e: Generated new TSK 0
I20260808 14:41:21.925571 32755 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:41:22.669440 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.144.84:49201/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35155
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:41:22.824064 318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:22.824383 318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:22.824429 318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:22.828946 318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:22.829071 318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:22.829183 318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:41:22.835101 318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.144.84:49201/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:22.836460 318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:22.837739 318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:22.853998 325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:22.854656 328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:22.855540 318 server_base.cc:1034] running on GCE node
W20260808 14:41:22.861877 326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:22.862690 318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:22.863708 318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:22.864890 318 hybrid_clock.cc:648] HybridClock initialized: now 1786200082864847 us; error 66 us; skew 500 ppm
Aug 08 14:41:22 dist-test-slave-p3t5 krb5kdc[32213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200082, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:22.869727 318 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:41:22.871065 318 webserver.cc:460] Webserver started at http://127.19.144.65:46753/ using document root <none> and password file <none>
I20260808 14:41:22.871927 318 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:22.872026 318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:22.872292 318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 14:41:22 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200082, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
I20260808 14:41:22.918718 318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4"
format_stamp: "Formatted at 2026-08-08 14:41:22 on dist-test-slave-p3t5"
server_key: "d0aaca437f27fc579bb0a552ca0b5419"
server_key_iv: "fae0922402f5c6c709dc7c4c782698e8"
server_key_version: "kuduclusterkey@0"
I20260808 14:41:22.919600 318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4"
format_stamp: "Formatted at 2026-08-08 14:41:22 on dist-test-slave-p3t5"
server_key: "d0aaca437f27fc579bb0a552ca0b5419"
server_key_iv: "fae0922402f5c6c709dc7c4c782698e8"
server_key_version: "kuduclusterkey@0"
I20260808 14:41:22.933899 318 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.007s sys 0.000s
I20260808 14:41:22.955701 337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:22.960270 318 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260808 14:41:22.960451 318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4"
format_stamp: "Formatted at 2026-08-08 14:41:22 on dist-test-slave-p3t5"
server_key: "d0aaca437f27fc579bb0a552ca0b5419"
server_key_iv: "fae0922402f5c6c709dc7c4c782698e8"
server_key_version: "kuduclusterkey@0"
I20260808 14:41:22.960626 318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:22.990227 318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:22.995182 318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:22.995473 318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:22.996402 318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:22.997716 318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:41:22.997786 318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:22.997860 318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:41:22.997900 318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:23.010735 318 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:33495
I20260808 14:41:23.010753 453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:33495 every 8 connection(s)
I20260808 14:41:23.012151 318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:41:23.016548 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 318
I20260808 14:41:23.016688 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:41:23.017056 20033 external_mini_cluster.cc:1428] Setting key fa80e069550dd67db19a8f78e0217e33
Aug 08 14:41:23 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200082, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:41:23.036201 457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:23.016371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:49717 (local address 127.19.144.126:35155)
0808 14:41:23.016660 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:23.016665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:23.019458 (+ 2793us) server_negotiation.cc:407] Connection header received
0808 14:41:23.019513 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:23.019516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:23.019579 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:23.019681 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:23.022592 (+ 2911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:23.023591 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:23.025839 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:23.026192 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:23.026282 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:23.030297 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:23.030303 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:23.030308 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:23.030353 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:23.032452 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:23.033975 (+ 1523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:23.033979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:23.033980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:23.034073 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:23.034601 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:23.034605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:23.034607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:23.034883 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:23.034994 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:23.035830 (+ 836us) server_negotiation.cc:299] Negotiation successful
0808 14:41:23.036002 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":85,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
I20260808 14:41:23.037707 456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:23.015009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35155 (local address 127.19.144.65:49717)
0808 14:41:23.017762 (+ 2753us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:23.017841 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:23.019102 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:23.020626 (+ 1524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:23.020636 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:23.021123 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:23.021993 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:23.022021 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:23.023815 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:23.023819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:23.025580 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:23.025596 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:23.026543 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:23.026547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:23.026769 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:23.027697 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:23.027715 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:23.030045 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:23.033303 (+ 3258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:23.033312 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:23.033347 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:23.033835 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:23.034184 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:23.034189 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:23.034191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:23.034470 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:23.035017 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:23.035025 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:23.035413 (+ 388us) client_negotiation.cc:769] Sending connection context
0808 14:41:23.035690 (+ 277us) client_negotiation.cc:240] Negotiation successful
0808 14:41:23.036377 (+ 687us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2461,"thread_start_us":140,"threads_started":1}
I20260808 14:41:23.040738 454 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35155
I20260808 14:41:23.041059 454 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:23.041786 454 heartbeater.cc:511] Master 127.19.144.126:35155 requested a full tablet report, sending...
Aug 08 14:41:23 dist-test-slave-p3t5 krb5kdc[32213](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786200083, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:23.043713 32703 ts_manager.cc:194] Registered new tserver with Master: e0e9c65b41be4dbd8aee9d693b5d48e4 (127.19.144.65:33495)
I20260808 14:41:23.045346 32703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:49717
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:41:23.063030 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.144.84:49201/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35155
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:41:23.228592 460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:23.229120 460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:23.229247 460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:23.235543 460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:23.235718 460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:23.235858 460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:41:23.244228 460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.144.84:49201/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:23.246042 460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:23.247603 460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:23.260933 469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:23.261440 466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:23.261780 467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:23.262655 460 server_base.cc:1034] running on GCE node
I20260808 14:41:23.263180 460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:23.264011 460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:23.265218 460 hybrid_clock.cc:648] HybridClock initialized: now 1786200083265181 us; error 49 us; skew 500 ppm
Aug 08 14:41:23 dist-test-slave-p3t5 krb5kdc[32213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200083, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:23.269279 460 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:41:23.270586 460 webserver.cc:460] Webserver started at http://127.19.144.66:39399/ using document root <none> and password file <none>
I20260808 14:41:23.271400 460 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:23.271526 460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:23.271803 460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 14:41:23 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200083, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
I20260808 14:41:23.291347 460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "443552cc40c24044bbb311f107b2311e"
format_stamp: "Formatted at 2026-08-08 14:41:23 on dist-test-slave-p3t5"
server_key: "88a0ece7bbafab1f144d9241e6f91219"
server_key_iv: "2467d11c70835670dc7019715cc6420d"
server_key_version: "kuduclusterkey@0"
I20260808 14:41:23.292286 460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "443552cc40c24044bbb311f107b2311e"
format_stamp: "Formatted at 2026-08-08 14:41:23 on dist-test-slave-p3t5"
server_key: "88a0ece7bbafab1f144d9241e6f91219"
server_key_iv: "2467d11c70835670dc7019715cc6420d"
server_key_version: "kuduclusterkey@0"
I20260808 14:41:23.298060 460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 14:41:23.325667 476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:23.327790 460 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.002s sys 0.000s
I20260808 14:41:23.327972 460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "443552cc40c24044bbb311f107b2311e"
format_stamp: "Formatted at 2026-08-08 14:41:23 on dist-test-slave-p3t5"
server_key: "88a0ece7bbafab1f144d9241e6f91219"
server_key_iv: "2467d11c70835670dc7019715cc6420d"
server_key_version: "kuduclusterkey@0"
I20260808 14:41:23.328146 460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:23.366617 460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:23.372542 460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:23.372769 460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:23.373515 460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:23.374655 460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:41:23.374727 460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:23.374796 460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:41:23.374830 460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:23.389773 460 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:36327
I20260808 14:41:23.391022 460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:41:23.391546 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 460
I20260808 14:41:23.391685 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:41:23.391932 592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:36327 every 8 connection(s)
I20260808 14:41:23.392091 20033 external_mini_cluster.cc:1428] Setting key a28ac6cd918581353e67b86bccd33833
Aug 08 14:41:23 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200083, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
I20260808 14:41:23.417445 457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:23.396789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:46565 (local address 127.19.144.126:35155)
0808 14:41:23.396969 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:23.396974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:23.397528 (+ 554us) server_negotiation.cc:407] Connection header received
0808 14:41:23.398816 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:23.398819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:23.398881 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:23.398983 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:23.401661 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:23.402482 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:23.404292 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:23.404657 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:23.404750 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:23.412258 (+ 7508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:23.412265 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:23.412272 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:23.412315 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:23.414304 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:23.415093 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:23.415098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:23.415100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:23.415203 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:23.415708 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:23.415713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:23.415715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:23.415989 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:23.416108 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:23.417008 (+ 900us) server_negotiation.cc:299] Negotiation successful
0808 14:41:23.417220 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
I20260808 14:41:23.420149 595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:23.395581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35155 (local address 127.19.144.66:46565)
0808 14:41:23.397307 (+ 1726us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:23.397376 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:23.398518 (+ 1142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:23.399348 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:23.399359 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:23.400639 (+ 1280us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:23.401409 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:23.401433 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:23.402670 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:23.402673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:23.404120 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:23.404132 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:23.405010 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:23.405014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:23.405246 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:23.408372 (+ 3126us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:23.408398 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:23.411948 (+ 3550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:23.414479 (+ 2531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:23.414488 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:23.414510 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:23.414949 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:23.415328 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:23.415331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:23.415333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:23.415565 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:23.416173 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:23.416181 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:23.416669 (+ 488us) client_negotiation.cc:769] Sending connection context
0808 14:41:23.418494 (+ 1825us) client_negotiation.cc:240] Negotiation successful
0808 14:41:23.418933 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1462,"thread_start_us":163,"threads_started":1}
I20260808 14:41:23.421694 593 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35155
I20260808 14:41:23.421991 593 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:23.422652 593 heartbeater.cc:511] Master 127.19.144.126:35155 requested a full tablet report, sending...
I20260808 14:41:23.423933 32703 ts_manager.cc:194] Registered new tserver with Master: 443552cc40c24044bbb311f107b2311e (127.19.144.66:36327)
I20260808 14:41:23.424765 32703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:46565
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:41:23.442106 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.144.84:49201/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35155
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:41:23 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200083, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
W20260808 14:41:23.612007 598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:23.612357 598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:23.612443 598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:23.616883 598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:23.616994 598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:23.617103 598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:41:23.623059 598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.144.84:49201/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:23.624366 598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:23.625844 598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:23.636590 605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:23.636898 607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:23.636590 604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:23.637702 598 server_base.cc:1034] running on GCE node
I20260808 14:41:23.638252 598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:23.639002 598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:23.640189 598 hybrid_clock.cc:648] HybridClock initialized: now 1786200083640149 us; error 53 us; skew 500 ppm
Aug 08 14:41:23 dist-test-slave-p3t5 krb5kdc[32213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200083, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:23.643421 598 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:41:23.644518 598 webserver.cc:460] Webserver started at http://127.19.144.67:43367/ using document root <none> and password file <none>
I20260808 14:41:23.645166 598 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:23.645269 598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:23.645515 598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 14:41:23 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200083, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
I20260808 14:41:23.665231 598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "9442897fcc4143bfa5b77e9602e888af"
format_stamp: "Formatted at 2026-08-08 14:41:23 on dist-test-slave-p3t5"
server_key: "8db90915509052a34be9c42b8ee7a95b"
server_key_iv: "a230e49e64dfbf7f874b0e1fd9b9bf92"
server_key_version: "kuduclusterkey@0"
I20260808 14:41:23.666946 598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "9442897fcc4143bfa5b77e9602e888af"
format_stamp: "Formatted at 2026-08-08 14:41:23 on dist-test-slave-p3t5"
server_key: "8db90915509052a34be9c42b8ee7a95b"
server_key_iv: "a230e49e64dfbf7f874b0e1fd9b9bf92"
server_key_version: "kuduclusterkey@0"
I20260808 14:41:23.672796 598 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 14:41:23.697294 614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:23.699251 598 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260808 14:41:23.699429 598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "9442897fcc4143bfa5b77e9602e888af"
format_stamp: "Formatted at 2026-08-08 14:41:23 on dist-test-slave-p3t5"
server_key: "8db90915509052a34be9c42b8ee7a95b"
server_key_iv: "a230e49e64dfbf7f874b0e1fd9b9bf92"
server_key_version: "kuduclusterkey@0"
I20260808 14:41:23.699594 598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:23.753357 598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:23.757664 598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:23.757915 598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:23.758783 598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:23.760071 598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:41:23.760146 598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:23.760211 598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:41:23.760298 598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260808 14:41:23.773885 598 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:37319
I20260808 14:41:23.773959 728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:37319 every 8 connection(s)
I20260808 14:41:23.775311 598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:41:23.776472 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 598
I20260808 14:41:23.776594 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:41:23.776911 20033 external_mini_cluster.cc:1428] Setting key a793233f7aba788961c3ee01a4cd8371
Aug 08 14:41:23 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200083, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:23.796083 457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:23.780683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:58169 (local address 127.19.144.126:35155)
0808 14:41:23.780819 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:23.780823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:23.780838 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:41:23.781724 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:23.781728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:23.781790 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:23.781850 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:23.783799 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:23.784677 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:23.786821 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:23.787174 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:23.787287 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:23.791192 (+ 3905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:23.791197 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:23.791201 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:23.791248 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:23.793416 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:23.794152 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:23.794156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:23.794158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:23.794255 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:23.794716 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:23.794719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:23.794721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:23.794974 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:23.795068 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:23.795787 (+ 719us) server_negotiation.cc:299] Negotiation successful
0808 14:41:23.795923 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:41:23.797999 731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:23.779498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35155 (local address 127.19.144.67:58169)
0808 14:41:23.780128 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:23.780196 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:23.781406 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:23.782344 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:23.782355 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:23.782781 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:23.783598 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:23.783622 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:23.784876 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:23.784880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:23.786588 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:23.786602 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:23.787485 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:23.787489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:23.787714 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:23.788472 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:23.788494 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:23.790936 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:23.793559 (+ 2623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:23.793566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:23.793587 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:23.794022 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:23.794378 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:23.794380 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:23.794382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:23.794589 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:23.795140 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:23.795147 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:23.795493 (+ 346us) client_negotiation.cc:769] Sending connection context
0808 14:41:23.796330 (+ 837us) client_negotiation.cc:240] Negotiation successful
0808 14:41:23.796704 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":167,"threads_started":1}
I20260808 14:41:23.799577 729 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35155
I20260808 14:41:23.799904 729 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:23.800594 729 heartbeater.cc:511] Master 127.19.144.126:35155 requested a full tablet report, sending...
I20260808 14:41:23.801839 32703 ts_manager.cc:194] Registered new tserver with Master: 9442897fcc4143bfa5b77e9602e888af (127.19.144.67:37319)
I20260808 14:41:23.802620 32703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:58169
I20260808 14:41:23.805047 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:41:23.805819 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 318
I20260808 14:41:23.821087 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 460
I20260808 14:41:23.832504 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 598
I20260808 14:41:23.843758 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 32670
I20260808 14:41:23.854583 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:35155
--webserver_interface=127.19.144.126
--webserver_port=44475
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:35155
--ranger_config_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.144.84:49201/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.144.84:49201/kms with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:41:23.981329 733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:23.981684 733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:23.981727 733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:23.986816 733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:41:23.986991 733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:23.987044 733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:23.987071 733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:41:23.987104 733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:41:23.993495 733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.144.84:49201/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:35155
--ranger_config_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:35155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=44475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:23.994757 733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:23.995908 733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:24.005205 741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:24.005272 733 server_base.cc:1034] running on GCE node
W20260808 14:41:24.005229 738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:24.005216 739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:24.006269 733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:24.007524 733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:24.008731 733 hybrid_clock.cc:648] HybridClock initialized: now 1786200084008697 us; error 54 us; skew 500 ppm
Aug 08 14:41:24 dist-test-slave-p3t5 krb5kdc[32213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200084, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:24.011683 733 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:41:24.012673 733 webserver.cc:460] Webserver started at http://127.19.144.126:44475/ using document root <none> and password file <none>
I20260808 14:41:24.013260 733 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:24.013372 733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 14:41:24 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200084, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
I20260808 14:41:24.062711 733 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 14:41:24.065321 750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:24.066794 733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:41:24.066939 733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "01554fc3f1444290abe4fef578e78c2e"
format_stamp: "Formatted at 2026-08-08 14:41:21 on dist-test-slave-p3t5"
server_key: "8db7c820d701c2a7aba7fab28c289f7a"
server_key_iv: "c98d9bbd9f030c9946c2773022ea5c37"
server_key_version: "kuduclusterkey@0"
I20260808 14:41:24.067648 733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:24.088383 733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:24.091425 733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:24.091645 733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:24.099653 733 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:35155
I20260808 14:41:24.099680 802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:35155 every 8 connection(s)
I20260808 14:41:24.100863 733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:41:24.104171 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 733
I20260808 14:41:24.104631 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.144.84:49201/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:33495
--local_ip_for_outbound_sockets=127.19.144.65
--tserver_master_addrs=127.19.144.126:35155
--webserver_port=46753
--webserver_interface=127.19.144.65
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.144.84:49201/kms with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:41:24.105633 803 sys_catalog.cc:263] Verifying existing consensus state
I20260808 14:41:24.107555 803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e: Bootstrap starting.
I20260808 14:41:24.124861 803 log.cc:824] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:24.132000 803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:41:24.132438 803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e: Bootstrap complete.
I20260808 14:41:24.136682 803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01554fc3f1444290abe4fef578e78c2e" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35155 } }
I20260808 14:41:24.137125 803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01554fc3f1444290abe4fef578e78c2e, State: Initialized, Role: FOLLOWER
I20260808 14:41:24.137732 803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01554fc3f1444290abe4fef578e78c2e" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35155 } }
I20260808 14:41:24.137883 803 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:41:24.137960 803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:41:24.138062 803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 1 FOLLOWER]: Advancing to term 2
I20260808 14:41:24.140133 803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01554fc3f1444290abe4fef578e78c2e" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35155 } }
I20260808 14:41:24.140486 803 leader_election.cc:302] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01554fc3f1444290abe4fef578e78c2e; no voters:
I20260808 14:41:24.140821 803 leader_election.cc:288] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 14:41:24.140983 808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 2 FOLLOWER]: Leader election won for term 2
I20260808 14:41:24.141230 808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [term 2 LEADER]: Becoming Leader. State: Replica: 01554fc3f1444290abe4fef578e78c2e, State: Running, Role: LEADER
I20260808 14:41:24.141561 808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01554fc3f1444290abe4fef578e78c2e" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35155 } }
I20260808 14:41:24.142158 803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:41:24.143113 810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "01554fc3f1444290abe4fef578e78c2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01554fc3f1444290abe4fef578e78c2e" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35155 } } }
I20260808 14:41:24.143247 810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [sys.catalog]: This master's current role is: LEADER
I20260808 14:41:24.143558 811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01554fc3f1444290abe4fef578e78c2e. Latest consensus state: current_term: 2 leader_uuid: "01554fc3f1444290abe4fef578e78c2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01554fc3f1444290abe4fef578e78c2e" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35155 } } }
I20260808 14:41:24.143647 811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e [sys.catalog]: This master's current role is: LEADER
I20260808 14:41:24.144124 818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:41:24.144584 803 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 14:41:24.150532 818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:41:24.151789 818 catalog_manager.cc:1259] Loaded cluster ID: 654c797ea63e47b685c61025436a974b
I20260808 14:41:24.151859 818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:41:24.154801 818 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:41:24.155803 818 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e: Loaded TSK: 0
I20260808 14:41:24.157282 818 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 14:41:24.258638 805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:24.259037 805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:24.259109 805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:24.265265 805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:24.265374 805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:24.265475 805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:41:24.273820 805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.144.84:49201/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:33495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=46753
--tserver_master_addrs=127.19.144.126:35155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:24.275382 805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:24.276959 805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:24.290665 846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:24.290665 844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:24.291330 805 server_base.cc:1034] running on GCE node
W20260808 14:41:24.290829 843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:24.291889 805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:24.292804 805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:24.296082 805 hybrid_clock.cc:648] HybridClock initialized: now 1786200084296045 us; error 50 us; skew 500 ppm
Aug 08 14:41:24 dist-test-slave-p3t5 krb5kdc[32213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200084, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:24.300289 805 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:41:24.301682 805 webserver.cc:460] Webserver started at http://127.19.144.65:46753/ using document root <none> and password file <none>
I20260808 14:41:24.302523 805 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:24.302688 805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 14:41:24 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200084, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
I20260808 14:41:24.334277 805 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:41:24.344672 854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:24.346971 805 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.001s
I20260808 14:41:24.347235 805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4"
format_stamp: "Formatted at 2026-08-08 14:41:22 on dist-test-slave-p3t5"
server_key: "d0aaca437f27fc579bb0a552ca0b5419"
server_key_iv: "fae0922402f5c6c709dc7c4c782698e8"
server_key_version: "kuduclusterkey@0"
I20260808 14:41:24.348201 805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:24.378925 805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:24.383149 805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:24.383390 805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:24.384196 805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:24.385347 805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:41:24.385414 805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:24.385484 805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:41:24.385509 805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:24.398715 805 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:33495
I20260808 14:41:24.398774 970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:33495 every 8 connection(s)
I20260808 14:41:24.400206 805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:41:24.407835 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 805
I20260808 14:41:24.408372 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.144.84:49201/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:36327
--local_ip_for_outbound_sockets=127.19.144.66
--tserver_master_addrs=127.19.144.126:35155
--webserver_port=39399
--webserver_interface=127.19.144.66
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.144.84:49201/kms with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:41:24 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200084, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:24.427086 973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:24.403993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:49773 (local address 127.19.144.126:35155)
0808 14:41:24.404558 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:24.404579 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:24.407164 (+ 2585us) server_negotiation.cc:407] Connection header received
0808 14:41:24.407551 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:24.407568 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:24.408181 (+ 613us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:24.408855 (+ 674us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:24.410778 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:24.412392 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:24.414067 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:24.414822 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:24.414992 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:24.419114 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:24.419122 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:24.419142 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:24.419179 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:24.423240 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:24.423900 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:24.423907 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:24.423912 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:24.424047 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:24.424442 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:24.424449 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:24.424451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:24.424897 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:24.425041 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:24.425743 (+ 702us) server_negotiation.cc:299] Negotiation successful
0808 14:41:24.426020 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":165,"threads_started":1}
I20260808 14:41:24.427098 974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:24.404184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35155 (local address 127.19.144.65:49773)
0808 14:41:24.405481 (+ 1297us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:24.405545 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:24.406785 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:24.409412 (+ 2627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:24.409423 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:24.409828 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:24.410557 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:24.410579 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:24.412572 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:24.412574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:24.413896 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:24.413909 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:24.415232 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:24.415236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:24.415445 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:24.416202 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:24.416224 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:24.418822 (+ 2598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:24.423400 (+ 4578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:24.423408 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:24.423429 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:24.423742 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:24.424157 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:24.424161 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:24.424163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:24.424328 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:24.425093 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:24.425099 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:24.425362 (+ 263us) client_negotiation.cc:769] Sending connection context
0808 14:41:24.425569 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 14:41:24.425992 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1061,"thread_start_us":163,"threads_started":1}
I20260808 14:41:24.430074 971 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35155
I20260808 14:41:24.430361 971 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:24.431016 971 heartbeater.cc:511] Master 127.19.144.126:35155 requested a full tablet report, sending...
W20260808 14:41:24.433171 971 heartbeater.cc:644] Failed to heartbeat to 127.19.144.126:35155 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 14:41:24.433313 971 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:24.433506 971 heartbeater.cc:511] Master 127.19.144.126:35155 requested a full tablet report, sending...
I20260808 14:41:24.434424 971 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:24.434631 971 heartbeater.cc:511] Master 127.19.144.126:35155 requested a full tablet report, sending...
W20260808 14:41:24.435482 971 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 14:41:24.557716 975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:24.558256 975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:24.558419 975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:24.565892 975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:24.566064 975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:24.566231 975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:41:24.573585 975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.144.84:49201/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:36327
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=39399
--tserver_master_addrs=127.19.144.126:35155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:24.574868 975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:24.576277 975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:24.589339 984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:24.589358 980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:24.589403 981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:24.590843 975 server_base.cc:1034] running on GCE node
I20260808 14:41:24.591564 975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:24.592509 975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:24.595826 975 hybrid_clock.cc:648] HybridClock initialized: now 1786200084595787 us; error 47 us; skew 500 ppm
Aug 08 14:41:24 dist-test-slave-p3t5 krb5kdc[32213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200084, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:24.600265 975 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:41:24.601588 975 webserver.cc:460] Webserver started at http://127.19.144.66:39399/ using document root <none> and password file <none>
I20260808 14:41:24.602496 975 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:24.602689 975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 14:41:24 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200084, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
I20260808 14:41:24.630075 975 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.004s sys 0.000s
I20260808 14:41:24.633301 992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:24.635246 975 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:41:24.635499 975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "443552cc40c24044bbb311f107b2311e"
format_stamp: "Formatted at 2026-08-08 14:41:23 on dist-test-slave-p3t5"
server_key: "88a0ece7bbafab1f144d9241e6f91219"
server_key_iv: "2467d11c70835670dc7019715cc6420d"
server_key_version: "kuduclusterkey@0"
I20260808 14:41:24.636373 975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:24.669422 975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:24.673018 975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:24.673243 975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:24.674018 975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:24.675114 975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:41:24.675189 975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:24.675263 975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:41:24.675314 975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:24.686635 975 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:36327
I20260808 14:41:24.686656 1108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:36327 every 8 connection(s)
I20260808 14:41:24.687847 975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:41:24.695812 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 975
I20260808 14:41:24.696409 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.144.84:49201/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:37319
--local_ip_for_outbound_sockets=127.19.144.67
--tserver_master_addrs=127.19.144.126:35155
--webserver_port=43367
--webserver_interface=127.19.144.67
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.144.84:49201/kms with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:41:24 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200084, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:24.710614 973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:24.691843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:37881 (local address 127.19.144.126:35155)
0808 14:41:24.691997 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:24.692002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:24.695236 (+ 3234us) server_negotiation.cc:407] Connection header received
0808 14:41:24.696549 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:24.696554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:24.696631 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:24.696694 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:24.698472 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:24.699083 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:24.700649 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:24.700900 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:24.701029 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:24.704899 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:24.704907 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:24.704912 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:24.704955 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:24.707171 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:24.708089 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:24.708094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:24.708096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:24.708221 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:24.708785 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:24.708789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:24.708792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:24.709089 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:24.709216 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:24.710159 (+ 943us) server_negotiation.cc:299] Negotiation successful
0808 14:41:24.710356 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 14:41:24.712818 1111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:24.690631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35155 (local address 127.19.144.66:37881)
0808 14:41:24.695034 (+ 4403us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:24.695092 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:24.696221 (+ 1129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:24.697187 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:24.697195 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:24.697598 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:24.698254 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:24.698288 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:24.699232 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:24.699234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:24.700505 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:24.700516 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:24.701496 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:24.701499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:24.701745 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:24.702497 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:24.702523 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:24.704623 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:24.707400 (+ 2777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:24.707412 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:24.707438 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:24.707921 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:24.708365 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:24.708369 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:24.708372 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:24.708636 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:24.709232 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:24.709244 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:24.709730 (+ 486us) client_negotiation.cc:769] Sending connection context
0808 14:41:24.710937 (+ 1207us) client_negotiation.cc:240] Negotiation successful
0808 14:41:24.711425 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4161,"thread_start_us":165,"threads_started":1}
I20260808 14:41:24.714499 1109 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35155
I20260808 14:41:24.714787 1109 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:24.715441 1109 heartbeater.cc:511] Master 127.19.144.126:35155 requested a full tablet report, sending...
W20260808 14:41:24.717506 1109 heartbeater.cc:644] Failed to heartbeat to 127.19.144.126:35155 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 14:41:24.717685 1109 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:24.717886 1109 heartbeater.cc:511] Master 127.19.144.126:35155 requested a full tablet report, sending...
I20260808 14:41:24.718924 1109 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:24.719110 1109 heartbeater.cc:511] Master 127.19.144.126:35155 requested a full tablet report, sending...
W20260808 14:41:24.720041 1109 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 14:41:24.851840 1112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:24.852171 1112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:24.852255 1112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:24.856961 1112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:24.857055 1112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:24.857159 1112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:41:24.865031 1112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.144.84:49201/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:37319
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=43367
--tserver_master_addrs=127.19.144.126:35155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:24.866629 1112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:24.868114 1112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:24.882866 1118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:24.883267 1112 server_base.cc:1034] running on GCE node
W20260808 14:41:24.883472 1121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:24.883584 1117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:24.884119 1112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:24.885066 1112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:24.886281 1112 hybrid_clock.cc:648] HybridClock initialized: now 1786200084886248 us; error 50 us; skew 500 ppm
Aug 08 14:41:24 dist-test-slave-p3t5 krb5kdc[32213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200084, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:24.890251 1112 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:41:24.891533 1112 webserver.cc:460] Webserver started at http://127.19.144.67:43367/ using document root <none> and password file <none>
I20260808 14:41:24.892414 1112 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:24.892542 1112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 14:41:24 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200084, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
I20260808 14:41:24.932451 1112 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:41:24.938560 1129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:24.940361 1112 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260808 14:41:24.940531 1112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "9442897fcc4143bfa5b77e9602e888af"
format_stamp: "Formatted at 2026-08-08 14:41:23 on dist-test-slave-p3t5"
server_key: "8db90915509052a34be9c42b8ee7a95b"
server_key_iv: "a230e49e64dfbf7f874b0e1fd9b9bf92"
server_key_version: "kuduclusterkey@0"
I20260808 14:41:24.941161 1112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:24.972515 1112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:24.975994 1112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:24.976227 1112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:24.976899 1112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:24.977885 1112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:41:24.977938 1112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:24.978012 1112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:41:24.978052 1112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:24.989552 1112 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:37319
I20260808 14:41:24.991750 1112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:41:24.990337 1250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:37319 every 8 connection(s)
I20260808 14:41:24.999521 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 1112
Aug 08 14:41:25 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200084, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:25.010111 1254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:25.000282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34230 (local address 127.19.144.126:35155)
0808 14:41:25.000610 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:25.000620 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:25.001045 (+ 425us) server_negotiation.cc:407] Connection header received
0808 14:41:25.001083 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:25.001085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:25.001125 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:25.001228 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:25.002247 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.002979 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.004301 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.004542 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.004647 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:25.006276 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:25.006280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:25.006283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:25.006311 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:25.008263 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:25.008721 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:25.008726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:25.008727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:25.008819 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:25.009120 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:25.009124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:25.009126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:25.009326 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:25.009566 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:25.009761 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 14:41:25.009892 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":121,"threads_started":1}
I20260808 14:41:25.011608 973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:24.994739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:47499 (local address 127.19.144.126:35155)
0808 14:41:24.994889 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:24.994893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:24.996043 (+ 1150us) server_negotiation.cc:407] Connection header received
0808 14:41:24.997305 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:24.997309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:24.997380 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:24.997491 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:24.998749 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:24.999381 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.001203 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.001472 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.001584 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:25.006771 (+ 5187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:25.006774 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:25.006778 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:25.006831 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:25.008711 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:25.009511 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:25.009515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:25.009517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:25.009610 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:25.010141 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:25.010145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:25.010147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:25.010391 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:25.010478 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:25.011319 (+ 841us) server_negotiation.cc:299] Negotiation successful
0808 14:41:25.011438 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:41:25.012892 1253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:24.995286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35155 (local address 127.19.144.67:47499)
0808 14:41:24.995852 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:24.995899 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:24.997040 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:24.997729 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:24.997736 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:24.998034 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:24.998597 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:24.998614 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:24.999549 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:24.999553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:25.001057 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:25.001070 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.001663 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.001668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:25.001863 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:25.002565 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:25.002589 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:25.006069 (+ 3480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:25.008879 (+ 2810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:25.008888 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:25.008910 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:25.009379 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:25.009772 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:25.009775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:25.009777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:25.010008 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:25.010575 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:25.010583 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:25.010973 (+ 390us) client_negotiation.cc:769] Sending connection context
0808 14:41:25.011279 (+ 306us) client_negotiation.cc:240] Negotiation successful
0808 14:41:25.011678 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":172,"threads_started":1}
I20260808 14:41:25.014207 1251 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35155
I20260808 14:41:25.014496 1251 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:25.015208 1251 heartbeater.cc:511] Master 127.19.144.126:35155 requested a full tablet report, sending...
I20260808 14:41:25.016680 764 ts_manager.cc:194] Registered new tserver with Master: 9442897fcc4143bfa5b77e9602e888af (127.19.144.67:37319)
I20260808 14:41:25.018033 764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:47499
Aug 08 14:41:25 dist-test-slave-p3t5 krb5kdc[32213](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786200085, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:25.435750 971 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:25.436110 971 heartbeater.cc:511] Master 127.19.144.126:35155 requested a full tablet report, sending...
I20260808 14:41:25.437018 764 ts_manager.cc:194] Registered new tserver with Master: e0e9c65b41be4dbd8aee9d693b5d48e4 (127.19.144.65:33495)
I20260808 14:41:25.437915 764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:49773
I20260808 14:41:25.720301 1109 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:25.720635 1109 heartbeater.cc:511] Master 127.19.144.126:35155 requested a full tablet report, sending...
I20260808 14:41:25.721551 764 ts_manager.cc:194] Registered new tserver with Master: 443552cc40c24044bbb311f107b2311e (127.19.144.66:36327)
I20260808 14:41:25.722150 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:41:25.722381 764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:37881
I20260808 14:41:25.733090 1267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:25.724506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37244 (local address 127.19.144.126:35155)
0808 14:41:25.725163 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:25.725168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:25.725181 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:41:25.725221 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:25.725223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:25.725266 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:25.725360 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:25.726141 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.726815 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.727672 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.727914 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.728026 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:25.729528 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:25.729532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:25.729535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:25.729565 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:25.731307 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:25.731764 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:25.731768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:25.731770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:25.731862 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:25.732151 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:25.732154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:25.732156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:25.732357 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:25.732501 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:25.732749 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 14:41:25.732884 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":106,"threads_started":1}
Aug 08 14:41:25 dist-test-slave-p3t5 krb5kdc[32213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200085, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for HTTP/127.19.144.84@KRBTEST.COM
I20260808 14:41:25.738327 764 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:41:25.743582 764 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:41:25.767849 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:25.758799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:37319 (local address 127.0.0.1:45554)
0808 14:41:25.759821 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:25.759834 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:25.759917 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:25.762494 (+ 2577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:25.762501 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:25.762529 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:25.762708 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:25.762720 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.764356 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.764362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:25.765876 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:25.765886 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.767255 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.767258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:25.767377 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:25.767419 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 14:41:25.767552 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 14:41:25.767681 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":958,"thread_start_us":124,"threads_started":1}
I20260808 14:41:25.770025 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:25.759289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:36327 (local address 127.0.0.1:36632)
0808 14:41:25.764542 (+ 5253us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:25.764559 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:25.764645 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:25.765550 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:25.765554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:25.765570 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:25.765683 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:25.765689 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.767120 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.767124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:25.768240 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:25.768252 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.769551 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.769555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:25.769679 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:25.769704 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 14:41:25.769778 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 14:41:25.769858 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5143,"thread_start_us":85,"threads_started":1}
I20260808 14:41:25.770543 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:25.760209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36632 (local address 127.19.144.66:36327)
0808 14:41:25.764900 (+ 4691us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:25.764907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:25.764944 (+ 37us) server_negotiation.cc:407] Connection header received
0808 14:41:25.765026 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:25.765031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:25.765255 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:25.765403 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:25.765818 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.766995 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.768388 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.769221 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.769331 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:25.769482 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:25.770279 (+ 797us) server_negotiation.cc:299] Negotiation successful
0808 14:41:25.770402 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4572,"thread_start_us":83,"threads_started":1}
I20260808 14:41:25.771682 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:25.758968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45554 (local address 127.19.144.67:37319)
0808 14:41:25.761868 (+ 2900us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:25.761876 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:25.761903 (+ 27us) server_negotiation.cc:407] Connection header received
0808 14:41:25.762000 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:25.762006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:25.762279 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:25.762903 (+ 624us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:25.763079 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.764224 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.766017 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.766896 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.767008 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:25.767197 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:25.771390 (+ 4193us) server_negotiation.cc:299] Negotiation successful
0808 14:41:25.771532 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2777,"thread_start_us":123,"threads_started":1}
I20260808 14:41:25.774355 1041 tablet_service.cc:1467] Processing CreateTablet for tablet 249408f3b98d4356a3b29abf982e0b91 (DEFAULT_TABLE table=test-table [id=e93e506dbd3748a7b805c26b01e42b1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:41:25.774802 1167 tablet_service.cc:1467] Processing CreateTablet for tablet 249408f3b98d4356a3b29abf982e0b91 (DEFAULT_TABLE table=test-table [id=e93e506dbd3748a7b805c26b01e42b1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:41:25.775647 1041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 249408f3b98d4356a3b29abf982e0b91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:25.775833 1167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 249408f3b98d4356a3b29abf982e0b91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:25.790133 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:25.758447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:33495 (local address 127.0.0.1:39320)
0808 14:41:25.759386 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:25.759443 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:25.759585 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:25.777721 (+ 18136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:25.777726 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:25.777761 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:25.777903 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:25.777911 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.787154 (+ 9243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.787158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:25.788344 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:25.788362 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.789607 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.789653 (+ 46us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:25.789768 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:25.789806 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 14:41:25.789866 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 14:41:25.789961 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"thread_start_us":116,"threads_started":1}
I20260808 14:41:25.791370 1277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:25.759845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39320 (local address 127.19.144.65:33495)
0808 14:41:25.772365 (+ 12520us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:25.772373 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:25.772405 (+ 32us) server_negotiation.cc:407] Connection header received
0808 14:41:25.772492 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:25.772499 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:25.772744 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:25.772980 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:25.785713 (+ 12733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.786956 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.788543 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.789337 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.790843 (+ 1506us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:25.791040 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:25.791103 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 14:41:25.791216 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12388,"thread_start_us":84,"threads_started":1}
I20260808 14:41:25.792904 1279 tablet_bootstrap.cc:492] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e: Bootstrap starting.
I20260808 14:41:25.795042 903 tablet_service.cc:1467] Processing CreateTablet for tablet 249408f3b98d4356a3b29abf982e0b91 (DEFAULT_TABLE table=test-table [id=e93e506dbd3748a7b805c26b01e42b1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:41:25.795610 1279 tablet_bootstrap.cc:654] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:25.796283 903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 249408f3b98d4356a3b29abf982e0b91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:25.796550 1279 log.cc:824] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:25.797407 1280 tablet_bootstrap.cc:492] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af: Bootstrap starting.
I20260808 14:41:25.799911 1280 tablet_bootstrap.cc:654] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:25.802278 1280 log.cc:824] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:25.805724 1283 tablet_bootstrap.cc:492] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4: Bootstrap starting.
I20260808 14:41:25.808481 1283 tablet_bootstrap.cc:654] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:25.809237 1279 tablet_bootstrap.cc:492] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e: No bootstrap required, opened a new log
I20260808 14:41:25.809495 1279 ts_tablet_manager.cc:1397] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.000s
I20260808 14:41:25.809530 1283 log.cc:824] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:25.813233 1279 raft_consensus.cc:348] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33495 } } peers { permanent_uuid: "443552cc40c24044bbb311f107b2311e" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36327 } } peers { permanent_uuid: "9442897fcc4143bfa5b77e9602e888af" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 37319 } }
I20260808 14:41:25.813494 1279 raft_consensus.cc:374] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:25.813539 1279 raft_consensus.cc:729] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 443552cc40c24044bbb311f107b2311e, State: Initialized, Role: FOLLOWER
I20260808 14:41:25.814204 1279 consensus_queue.cc:260] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33495 } } peers { permanent_uuid: "443552cc40c24044bbb311f107b2311e" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36327 } } peers { permanent_uuid: "9442897fcc4143bfa5b77e9602e888af" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 37319 } }
I20260808 14:41:25.815624 1279 ts_tablet_manager.cc:1428] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 14:41:25.816119 1109 heartbeater.cc:503] Master 127.19.144.126:35155 was elected leader, sending a full tablet report...
I20260808 14:41:25.817966 1280 tablet_bootstrap.cc:492] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af: No bootstrap required, opened a new log
I20260808 14:41:25.818195 1280 ts_tablet_manager.cc:1397] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.000s
I20260808 14:41:25.822981 1280 raft_consensus.cc:348] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33495 } } peers { permanent_uuid: "443552cc40c24044bbb311f107b2311e" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36327 } } peers { permanent_uuid: "9442897fcc4143bfa5b77e9602e888af" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 37319 } }
I20260808 14:41:25.823475 1280 raft_consensus.cc:374] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:25.823596 1280 raft_consensus.cc:729] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9442897fcc4143bfa5b77e9602e888af, State: Initialized, Role: FOLLOWER
I20260808 14:41:25.824895 1283 tablet_bootstrap.cc:492] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4: No bootstrap required, opened a new log
I20260808 14:41:25.825245 1283 ts_tablet_manager.cc:1397] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.000s
I20260808 14:41:25.825591 1280 consensus_queue.cc:260] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33495 } } peers { permanent_uuid: "443552cc40c24044bbb311f107b2311e" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36327 } } peers { permanent_uuid: "9442897fcc4143bfa5b77e9602e888af" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 37319 } }
I20260808 14:41:25.827013 1280 ts_tablet_manager.cc:1428] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af: Time spent starting tablet: real 0.009s user 0.004s sys 0.003s
I20260808 14:41:25.827476 1251 heartbeater.cc:503] Master 127.19.144.126:35155 was elected leader, sending a full tablet report...
I20260808 14:41:25.829739 1283 raft_consensus.cc:348] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33495 } } peers { permanent_uuid: "443552cc40c24044bbb311f107b2311e" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36327 } } peers { permanent_uuid: "9442897fcc4143bfa5b77e9602e888af" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 37319 } }
I20260808 14:41:25.830193 1283 raft_consensus.cc:374] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:25.830307 1283 raft_consensus.cc:729] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e9c65b41be4dbd8aee9d693b5d48e4, State: Initialized, Role: FOLLOWER
I20260808 14:41:25.830945 1283 consensus_queue.cc:260] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33495 } } peers { permanent_uuid: "443552cc40c24044bbb311f107b2311e" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36327 } } peers { permanent_uuid: "9442897fcc4143bfa5b77e9602e888af" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 37319 } }
I20260808 14:41:25.832310 1283 ts_tablet_manager.cc:1428] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 14:41:25.832362 971 heartbeater.cc:503] Master 127.19.144.126:35155 was elected leader, sending a full tablet report...
I20260808 14:41:25.870612 1287 raft_consensus.cc:482] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:41:25.870913 1287 raft_consensus.cc:504] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33495 } } peers { permanent_uuid: "443552cc40c24044bbb311f107b2311e" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36327 } } peers { permanent_uuid: "9442897fcc4143bfa5b77e9602e888af" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 37319 } }
I20260808 14:41:25.872401 1287 leader_election.cc:288] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 443552cc40c24044bbb311f107b2311e (127.19.144.66:36327), 9442897fcc4143bfa5b77e9602e888af (127.19.144.67:37319)
I20260808 14:41:25.877180 1289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:25.872718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:37319 (local address 127.19.144.65:42483)
0808 14:41:25.873238 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:25.873253 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:25.873384 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:25.873822 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:25.873825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:25.873840 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:25.873968 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:25.873974 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.874954 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.874957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:25.875899 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:25.875909 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.876692 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.876696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:25.876799 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:25.876826 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 14:41:25.876909 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 14:41:25.876985 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":45,"thread_start_us":129,"threads_started":1}
I20260808 14:41:25.877188 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:25.872989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:42483 (local address 127.19.144.67:37319)
0808 14:41:25.873126 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:25.873131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:25.873343 (+ 212us) server_negotiation.cc:407] Connection header received
0808 14:41:25.873555 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:25.873558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:25.873634 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:25.873711 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:25.874093 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.874822 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.876043 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.876561 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.876691 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:25.876751 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:25.876946 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 14:41:25.877051 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:41:25.877709 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:25.872981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:56533 (local address 127.19.144.66:36327)
0808 14:41:25.873126 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:25.873131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:25.873249 (+ 118us) server_negotiation.cc:407] Connection header received
0808 14:41:25.873454 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:25.873460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:25.873525 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:25.873646 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:25.873950 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.874792 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.876394 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.876980 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.877096 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:25.877145 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:25.877495 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 14:41:25.877578 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 14:41:25.877728 1288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:25.872665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:36327 (local address 127.19.144.65:56533)
0808 14:41:25.873131 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:25.873148 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:25.873286 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:25.873657 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:25.873660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:25.873676 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:25.873829 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:25.873835 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.874925 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.874929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:25.876250 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:25.876262 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.877212 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.877218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:25.877361 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:25.877392 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 14:41:25.877474 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 14:41:25.877550 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":50,"thread_start_us":182,"threads_started":1}
I20260808 14:41:25.878290 1204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "249408f3b98d4356a3b29abf982e0b91" candidate_uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9442897fcc4143bfa5b77e9602e888af" is_pre_election: true
I20260808 14:41:25.878289 1062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "249408f3b98d4356a3b29abf982e0b91" candidate_uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443552cc40c24044bbb311f107b2311e" is_pre_election: true
I20260808 14:41:25.878664 1204 raft_consensus.cc:2454] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0e9c65b41be4dbd8aee9d693b5d48e4 in term 0.
I20260808 14:41:25.878664 1062 raft_consensus.cc:2454] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0e9c65b41be4dbd8aee9d693b5d48e4 in term 0.
I20260808 14:41:25.879282 858 leader_election.cc:302] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9442897fcc4143bfa5b77e9602e888af, e0e9c65b41be4dbd8aee9d693b5d48e4; no voters:
I20260808 14:41:25.879671 1287 raft_consensus.cc:2781] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:41:25.879768 1287 raft_consensus.cc:482] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:41:25.879813 1287 raft_consensus.cc:3037] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:25.880950 1287 raft_consensus.cc:504] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33495 } } peers { permanent_uuid: "443552cc40c24044bbb311f107b2311e" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36327 } } peers { permanent_uuid: "9442897fcc4143bfa5b77e9602e888af" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 37319 } }
I20260808 14:41:25.881318 1287 leader_election.cc:288] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [CANDIDATE]: Term 1 election: Requested vote from peers 443552cc40c24044bbb311f107b2311e (127.19.144.66:36327), 9442897fcc4143bfa5b77e9602e888af (127.19.144.67:37319)
I20260808 14:41:25.881808 1062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "249408f3b98d4356a3b29abf982e0b91" candidate_uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443552cc40c24044bbb311f107b2311e"
I20260808 14:41:25.881944 1062 raft_consensus.cc:3037] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:25.883155 1062 raft_consensus.cc:2454] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0e9c65b41be4dbd8aee9d693b5d48e4 in term 1.
I20260808 14:41:25.883468 1204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "249408f3b98d4356a3b29abf982e0b91" candidate_uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9442897fcc4143bfa5b77e9602e888af"
I20260808 14:41:25.883630 1204 raft_consensus.cc:3037] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:25.883931 857 leader_election.cc:302] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 443552cc40c24044bbb311f107b2311e, e0e9c65b41be4dbd8aee9d693b5d48e4; no voters:
I20260808 14:41:25.884166 1287 raft_consensus.cc:2781] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:41:25.884423 1287 raft_consensus.cc:686] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [term 1 LEADER]: Becoming Leader. State: Replica: e0e9c65b41be4dbd8aee9d693b5d48e4, State: Running, Role: LEADER
I20260808 14:41:25.884817 1287 consensus_queue.cc:237] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33495 } } peers { permanent_uuid: "443552cc40c24044bbb311f107b2311e" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36327 } } peers { permanent_uuid: "9442897fcc4143bfa5b77e9602e888af" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 37319 } }
I20260808 14:41:25.885186 1204 raft_consensus.cc:2454] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0e9c65b41be4dbd8aee9d693b5d48e4 in term 1.
I20260808 14:41:25.888876 764 catalog_manager.cc:5495] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0e9c65b41be4dbd8aee9d693b5d48e4 (127.19.144.65). New cstate: current_term: 1 leader_uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e9c65b41be4dbd8aee9d693b5d48e4" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 33495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "443552cc40c24044bbb311f107b2311e" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9442897fcc4143bfa5b77e9602e888af" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 37319 } health_report { overall_health: UNKNOWN } } }
W20260808 14:41:25.905303 972 tablet.cc:2369] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:41:25.926363 1277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:25.921068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39324 (local address 127.19.144.65:33495)
0808 14:41:25.921249 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:25.921254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:25.921274 (+ 20us) server_negotiation.cc:407] Connection header received
0808 14:41:25.921383 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:25.921387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:25.921474 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:25.921659 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:41:25.921984 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.922811 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.924282 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:25.924658 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:25.924765 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:25.925180 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:41:25.925837 (+ 657us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:41:25.925968 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:25.926089 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 14:41:25.926167 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:41:25.934710 1204 raft_consensus.cc:1261] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af [term 1 FOLLOWER]: Refusing update from remote peer e0e9c65b41be4dbd8aee9d693b5d48e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:41:25.934811 1062 raft_consensus.cc:1261] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e [term 1 FOLLOWER]: Refusing update from remote peer e0e9c65b41be4dbd8aee9d693b5d48e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:41:25.935829 1290 consensus_queue.cc:1035] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9442897fcc4143bfa5b77e9602e888af" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 37319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:41:25.936157 1287 consensus_queue.cc:1035] T 249408f3b98d4356a3b29abf982e0b91 P e0e9c65b41be4dbd8aee9d693b5d48e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "443552cc40c24044bbb311f107b2311e" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 14:41:25.940390 1110 tablet.cc:2369] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:41:25.946789 1293 mvcc.cc:205] Tried to move back new op lower bound from 7316275551979061248 to 7316275551788228608. Current Snapshot: MvccSnapshot[applied={T|T < 7316275551979061248}]
I20260808 14:41:25.950022 1294 mvcc.cc:205] Tried to move back new op lower bound from 7316275551979061248 to 7316275551788228608. Current Snapshot: MvccSnapshot[applied={T|T < 7316275551979061248}]
I20260808 14:41:25.950374 1295 mvcc.cc:205] Tried to move back new op lower bound from 7316275551979061248 to 7316275551788228608. Current Snapshot: MvccSnapshot[applied={T|T < 7316275551979061248}]
I20260808 14:41:25.961063 765 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:41:25.961287 765 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:41:25.964478 765 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01554fc3f1444290abe4fef578e78c2e: Sending DeleteTablet for 3 replicas of tablet 249408f3b98d4356a3b29abf982e0b91
I20260808 14:41:25.966039 1041 tablet_service.cc:1514] Processing DeleteTablet for tablet 249408f3b98d4356a3b29abf982e0b91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:41:25 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:36632
I20260808 14:41:25.966749 1167 tablet_service.cc:1514] Processing DeleteTablet for tablet 249408f3b98d4356a3b29abf982e0b91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:41:25 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:45554
I20260808 14:41:25.967319 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 805
I20260808 14:41:25.973963 1303 tablet_replica.cc:330] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e: stopping tablet replica
I20260808 14:41:25.974469 1303 raft_consensus.cc:2229] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:41:25.974956 1303 raft_consensus.cc:2258] T 249408f3b98d4356a3b29abf982e0b91 P 443552cc40c24044bbb311f107b2311e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:41:25.975482 1304 tablet_replica.cc:330] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af: stopping tablet replica
I20260808 14:41:25.975873 1304 raft_consensus.cc:2229] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:41:25.976258 1304 raft_consensus.cc:2258] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:41:25.978391 1304 ts_tablet_manager.cc:1905] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 14:41:25.980266 753 connection.cc:535] client connection to 127.19.144.65:33495 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 14:41:25.980443 753 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 14:41:25.981837 1304 ts_tablet_manager.cc:1918] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:41:25.982354 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:25.981970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:33495 (local address 127.0.0.1:39330)
0808 14:41:25.982178 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:25.982253 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:33495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260808 14:41:25.982669 753 catalog_manager.cc:4584] TS e0e9c65b41be4dbd8aee9d693b5d48e4 (127.19.144.65:33495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 249408f3b98d4356a3b29abf982e0b91: Network error: Client connection negotiation failed: client connection to 127.19.144.65:33495: connect: Connection refused (error 111)
I20260808 14:41:25.983227 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 975
I20260808 14:41:25.983911 1304 log.cc:1196] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/249408f3b98d4356a3b29abf982e0b91
I20260808 14:41:25.984340 1304 ts_tablet_manager.cc:1939] T 249408f3b98d4356a3b29abf982e0b91 P 9442897fcc4143bfa5b77e9602e888af: Deleting consensus metadata
I20260808 14:41:25.986029 754 catalog_manager.cc:4841] TS 9442897fcc4143bfa5b77e9602e888af (127.19.144.67:37319): tablet 249408f3b98d4356a3b29abf982e0b91 (table test-table [id=e93e506dbd3748a7b805c26b01e42b1d]) successfully deleted
W20260808 14:41:26.001153 1273 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:41:25.998029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:36327 (local address 127.0.0.1:36638)
0808 14:41:25.998266 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:25.998283 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:25.998510 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:26.001027 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.66:36327: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":144}
I20260808 14:41:26.001976 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 1112
I20260808 14:41:26.012570 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 733
I20260808 14:41:26.021925 20033 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 14:41:26.368144 20033 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 14:41:26.368327 20033 mini_ranger.cc:66] Stopping Ranger...
I20260808 14:41:26.713948 20033 mini_ranger.cc:68] Stopped Ranger
2026-08-08 14:41:26.714 UTC [32243] LOG: received smart shutdown request
2026-08-08 14:41:26.718 UTC [32243] LOG: background worker "logical replication launcher" (PID 32253) exited with exit code 1
2026-08-08 14:41:26.719 UTC [32247] LOG: shutting down
2026-08-08 14:41:26.917 UTC [32243] LOG: database system is shut down
2026-08-08T14:41:26Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51076 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:41:27 dist-test-slave-p3t5 krb5kdc[1314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:41:27 dist-test-slave-p3t5 krb5kdc[1314](info): set up 2 sockets
Aug 08 14:41:27 dist-test-slave-p3t5 krb5kdc[1314](info): commencing operation
krb5kdc: starting...
W20260808 14:41:28.199788 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:41:28 dist-test-slave-p3t5 krb5kdc[1314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:41:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:41:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:41:28.342149 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:39547
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:39547,127.19.144.125:42683,127.19.144.124:41805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:41:28.467849 1327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:28.468199 1327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:28.468245 1327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:28.472572 1327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:41:28.472663 1327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:28.472684 1327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:28.472702 1327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:41:28.472718 1327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:41:28.478358 1327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:39547,127.19.144.125:42683,127.19.144.124:41805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:39547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:28.479602 1327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:28.480687 1327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:28.489823 1336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:28.489879 1333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:28.489879 1327 server_base.cc:1034] running on GCE node
W20260808 14:41:28.490128 1332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:28.490754 1327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:28.492048 1327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:28.493242 1327 hybrid_clock.cc:648] HybridClock initialized: now 1786200088493221 us; error 38 us; skew 500 ppm
Aug 08 14:41:28 dist-test-slave-p3t5 krb5kdc[1314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200088, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:28.496529 1327 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:41:28.497548 1327 webserver.cc:460] Webserver started at http://127.19.144.126:39339/ using document root <none> and password file <none>
I20260808 14:41:28.498190 1327 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:28.498256 1327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:28.498481 1327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:41:28.500705 1327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "1e8df0db57a14443ae2d9ef82831829f"
format_stamp: "Formatted at 2026-08-08 14:41:28 on dist-test-slave-p3t5"
server_key: "cbac885a7af93ecb362da9c646c90a45"
server_key_iv: "f1d338e0aa30426ddc2007f9f1390d1d"
server_key_version: "encryptionkey@0"
I20260808 14:41:28.501324 1327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "1e8df0db57a14443ae2d9ef82831829f"
format_stamp: "Formatted at 2026-08-08 14:41:28 on dist-test-slave-p3t5"
server_key: "cbac885a7af93ecb362da9c646c90a45"
server_key_iv: "f1d338e0aa30426ddc2007f9f1390d1d"
server_key_version: "encryptionkey@0"
I20260808 14:41:28.505275 1327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 14:41:28.508045 1343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:28.509465 1327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:41:28.509662 1327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "1e8df0db57a14443ae2d9ef82831829f"
format_stamp: "Formatted at 2026-08-08 14:41:28 on dist-test-slave-p3t5"
server_key: "cbac885a7af93ecb362da9c646c90a45"
server_key_iv: "f1d338e0aa30426ddc2007f9f1390d1d"
server_key_version: "encryptionkey@0"
I20260808 14:41:28.509804 1327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:28.537817 1327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:28.541718 1327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:28.541958 1327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:28.549748 1327 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:39547
I20260808 14:41:28.550196 1396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:39547 every 8 connection(s)
I20260808 14:41:28.550856 1327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:41:28.551155 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 1327
I20260808 14:41:28.551328 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:41:28.551715 20033 external_mini_cluster.cc:1428] Setting key e186a27050d314e11c0783ec6ce3206f
I20260808 14:41:28.554713 1397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:28.559909 1397 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.144.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.125@KRBTEST.COM" created.
Aug 08 14:41:28 dist-test-slave-p3t5 krb5kdc[1314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200088, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:28.580029 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.564451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51276 (local address 127.19.144.126:39547)
0808 14:41:28.564967 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:28.564979 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:28.565041 (+ 62us) server_negotiation.cc:407] Connection header received
0808 14:41:28.566851 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:28.566857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:28.567179 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:28.567408 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:28.568647 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:28.570022 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:28.571171 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:28.571607 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:28.571788 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:28.574549 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:28.574556 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:28.574567 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:28.574591 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:28.576382 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:28.576901 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:28.576913 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:28.576915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:28.577060 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:28.577409 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:28.577412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:28.577413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:28.578011 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:28.578156 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:28.578547 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 14:41:28.578843 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":156,"threads_started":1}
I20260808 14:41:28.580029 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.563210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:39547 (local address 127.0.0.1:51276)
0808 14:41:28.564064 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.564136 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:28.566383 (+ 2247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:28.567444 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:28.567454 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:28.568041 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:28.568442 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:28.568485 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:28.570163 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:28.570166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:28.571003 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:28.571014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:28.571786 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:28.571790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:28.571883 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:28.572578 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:28.572597 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:28.574370 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:28.576520 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:28.576532 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:28.576544 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:28.576785 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:28.577156 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:28.577161 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:28.577163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:28.577295 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:28.578134 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:28.578142 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:28.578297 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 14:41:28.578479 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 14:41:28.578933 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":110,"thread_start_us":190,"threads_started":1}
I20260808 14:41:28.582908 1397 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:41:28.583894 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.583499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:42683 (local address 127.0.0.1:53506)
0808 14:41:28.583693 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.583782 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260808 14:41:28.584203 1345 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.144.125:42683: connect: Connection refused (error 111)
I20260808 14:41:28.585132 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.584917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:42683 (local address 127.0.0.1:53518)
0808 14:41:28.585022 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.585069 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 14:41:28.585327 1397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.125:42683: Network error: Client connection negotiation failed: client connection to 127.19.144.125:42683: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.125.keytab.
Entry for principal kudu/127.19.144.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.125.keytab.
I20260808 14:41:28.586622 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.125:42683
--webserver_interface=127.19.144.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.125.keytab
--principal=kudu/127.19.144.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:39547,127.19.144.125:42683,127.19.144.124:41805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:41:28.634653 1397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } attempt: 1
I20260808 14:41:28.635708 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.635374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:42683 (local address 127.0.0.1:53522)
0808 14:41:28.635550 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.635622 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 14:41:28.636988 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.636424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:42683 (local address 127.0.0.1:53526)
0808 14:41:28.636848 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.636909 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260808 14:41:28.637217 1397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.125:42683: Network error: Client connection negotiation failed: client connection to 127.19.144.125:42683: connect: Connection refused (error 111)
I20260808 14:41:28.705425 1397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } attempt: 2
I20260808 14:41:28.706493 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.706079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:42683 (local address 127.0.0.1:53540)
0808 14:41:28.706287 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.706367 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 14:41:28.707355 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.707103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:42683 (local address 127.0.0.1:53556)
0808 14:41:28.707225 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.707285 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 14:41:28.707557 1397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.125:42683: Network error: Client connection negotiation failed: client connection to 127.19.144.125:42683: connect: Connection refused (error 111)
W20260808 14:41:28.711474 1404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:28.711786 1404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:28.711862 1404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:28.716285 1404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:41:28.716360 1404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:28.716380 1404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:28.716399 1404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:41:28.716437 1404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:41:28.722247 1404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:39547,127.19.144.125:42683,127.19.144.124:41805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.125.keytab
--principal=kudu/127.19.144.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.125:42683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.144.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:28.723546 1404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:28.724749 1404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:28.734165 1410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:28.734381 1413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:28.734378 1404 server_base.cc:1034] running on GCE node
W20260808 14:41:28.734215 1411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:28.735173 1404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:28.736462 1404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:28.737703 1404 hybrid_clock.cc:648] HybridClock initialized: now 1786200088737661 us; error 56 us; skew 500 ppm
Aug 08 14:41:28 dist-test-slave-p3t5 krb5kdc[1314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200088, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:28.740850 1404 init.cc:377] Logged in from keytab as kudu/127.19.144.125@KRBTEST.COM (short username kudu)
I20260808 14:41:28.741988 1404 webserver.cc:460] Webserver started at http://127.19.144.125:38893/ using document root <none> and password file <none>
I20260808 14:41:28.742642 1404 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:28.742743 1404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:28.743000 1404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:41:28.745340 1404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/data/instance:
uuid: "0f52265b04294161aa504efcacbca840"
format_stamp: "Formatted at 2026-08-08 14:41:28 on dist-test-slave-p3t5"
server_key: "00df064fb5c445d21694577d4a4149c5"
server_key_iv: "4e45e33bd36d56f66833108258956b6c"
server_key_version: "encryptionkey@0"
I20260808 14:41:28.746061 1404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/wal/instance:
uuid: "0f52265b04294161aa504efcacbca840"
format_stamp: "Formatted at 2026-08-08 14:41:28 on dist-test-slave-p3t5"
server_key: "00df064fb5c445d21694577d4a4149c5"
server_key_iv: "4e45e33bd36d56f66833108258956b6c"
server_key_version: "encryptionkey@0"
I20260808 14:41:28.750589 1404 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:41:28.753541 1420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:28.755183 1404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 14:41:28.755374 1404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/wal
uuid: "0f52265b04294161aa504efcacbca840"
format_stamp: "Formatted at 2026-08-08 14:41:28 on dist-test-slave-p3t5"
server_key: "00df064fb5c445d21694577d4a4149c5"
server_key_iv: "4e45e33bd36d56f66833108258956b6c"
server_key_version: "encryptionkey@0"
I20260808 14:41:28.755517 1404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:28.797082 1404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:28.798851 1397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } attempt: 3
I20260808 14:41:28.799773 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.799455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:42683 (local address 127.0.0.1:53568)
0808 14:41:28.799612 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.799684 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 14:41:28.800678 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.800405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:42683 (local address 127.0.0.1:53574)
0808 14:41:28.800544 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.800601 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 14:41:28.800954 1397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.125:42683: Network error: Client connection negotiation failed: client connection to 127.19.144.125:42683: connect: Connection refused (error 111)
I20260808 14:41:28.801231 1404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:28.801434 1404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:28.808964 1404 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.125:42683
I20260808 14:41:28.809291 1473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.125:42683 every 8 connection(s)
I20260808 14:41:28.810128 1404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/data/info.pb
I20260808 14:41:28.813710 1474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:28.816089 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 1404
I20260808 14:41:28.816241 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-1/wal/instance
I20260808 14:41:28.816536 20033 external_mini_cluster.cc:1428] Setting key 2af52c659fee6ff83cbe7d57606b63ef
I20260808 14:41:28.818473 1474 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.144.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.124@KRBTEST.COM" created.
Aug 08 14:41:28 dist-test-slave-p3t5 krb5kdc[1314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200088, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.125@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:28.835198 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.820743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51278 (local address 127.19.144.126:39547)
0808 14:41:28.820922 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:28.820927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:28.822015 (+ 1088us) server_negotiation.cc:407] Connection header received
0808 14:41:28.823107 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:28.823113 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:28.823196 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:28.823330 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:28.824626 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:28.825532 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:28.826986 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:28.827251 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:28.827360 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:28.830718 (+ 3358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:28.830721 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:28.830723 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:28.830753 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:28.832608 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:28.833291 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:28.833295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:28.833298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:28.833416 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:28.833893 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:28.833896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:28.833898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:28.834115 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:28.834229 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:28.834861 (+ 632us) server_negotiation.cc:299] Negotiation successful
0808 14:41:28.835010 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 14:41:28.836114 1477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.821231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:39547 (local address 127.0.0.1:51278)
0808 14:41:28.821824 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.821885 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:28.822856 (+ 971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:28.823510 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:28.823518 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:28.823897 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:28.824440 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:28.824456 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:28.825761 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:28.825768 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:28.826823 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:28.826836 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:28.827423 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:28.827426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:28.827595 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:28.828260 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:28.828281 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:28.830499 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:28.832790 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:28.832797 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:28.832813 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:28.833172 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:28.833534 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:28.833538 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:28.833540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:28.833787 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:28.834231 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:28.834241 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:28.834556 (+ 315us) client_negotiation.cc:769] Sending connection context
0808 14:41:28.834759 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 14:41:28.835148 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":171,"threads_started":1}
I20260808 14:41:28.838017 1474 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:41:28 dist-test-slave-p3t5 krb5kdc[1314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200088, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.125@KRBTEST.COM for kudu/127.19.144.125@KRBTEST.COM
I20260808 14:41:28.847864 1477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.838553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:42683 (local address 127.0.0.1:53586)
0808 14:41:28.838679 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.838697 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:28.838772 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:28.839395 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:28.839398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:28.839593 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:28.839673 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:28.839679 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:28.840656 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:28.840662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:28.841211 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:28.841219 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:28.841908 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:28.841911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:28.842005 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:28.842525 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:28.842543 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:28.844018 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:28.846167 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:28.846170 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:28.846172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:28.846379 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:28.846696 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:28.846699 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:28.846701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:28.846794 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:28.847285 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:28.847288 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:28.847387 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 14:41:28.847499 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 14:41:28.847656 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 14:41:28.847898 1479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.838703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53586 (local address 127.19.144.125:42683)
0808 14:41:28.839016 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:28.839025 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:28.839044 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:41:28.839079 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:28.839083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:28.839245 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:28.839426 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:28.839782 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:28.840539 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:28.841328 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:28.841790 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:28.841907 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:28.844166 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:28.844178 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:28.844200 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:28.844221 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:28.846038 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:28.846500 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:28.846506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:28.846507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:28.846588 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:28.846885 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:28.846890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:28.846891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:28.847171 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:28.847318 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:28.847606 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 14:41:28.847745 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.124.keytab.
Entry for principal kudu/127.19.144.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.124.keytab.
I20260808 14:41:28.850082 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.124:41805
--webserver_interface=127.19.144.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.124.keytab
--principal=kudu/127.19.144.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:39547,127.19.144.125:42683,127.19.144.124:41805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:41:28.850395 1474 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:41:28.851260 1477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.850945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:41805 (local address 127.0.0.1:44308)
0808 14:41:28.851083 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.851177 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:41805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 14:41:28.851610 1422 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.144.124:41805: connect: Connection refused (error 111)
I20260808 14:41:28.852910 1477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.852548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:41805 (local address 127.0.0.1:44310)
0808 14:41:28.852687 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.852756 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:41805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 14:41:28.853091 1474 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:41805: Network error: Client connection negotiation failed: client connection to 127.19.144.124:41805: connect: Connection refused (error 111)
I20260808 14:41:28.902405 1474 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } attempt: 1
I20260808 14:41:28.903424 1477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.903097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:41805 (local address 127.0.0.1:44320)
0808 14:41:28.903264 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.903336 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:41805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 14:41:28.904239 1477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.904013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:41805 (local address 127.0.0.1:44336)
0808 14:41:28.904123 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.904176 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:41805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 14:41:28.904459 1474 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:41805: Network error: Client connection negotiation failed: client connection to 127.19.144.124:41805: connect: Connection refused (error 111)
I20260808 14:41:28.944204 1397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } attempt: 4
Aug 08 14:41:28 dist-test-slave-p3t5 krb5kdc[1314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200088, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for kudu/127.19.144.125@KRBTEST.COM
I20260808 14:41:28.955304 1479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.945010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53590 (local address 127.19.144.125:42683)
0808 14:41:28.945182 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:28.945187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:28.945342 (+ 155us) server_negotiation.cc:407] Connection header received
0808 14:41:28.945468 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:28.945471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:28.945523 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:28.945685 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:28.946206 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:28.946758 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:28.947926 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:28.948732 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:28.948852 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:28.951587 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:28.951591 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:28.951593 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:28.951621 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:28.953323 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:28.953908 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:28.953910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:28.953912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:28.953996 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:28.954355 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:28.954358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:28.954359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:28.954602 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:28.954689 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:28.955053 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 14:41:28.955172 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 14:41:28.955382 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.944953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:42683 (local address 127.0.0.1:53590)
0808 14:41:28.945205 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.945224 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:28.945365 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:28.945684 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:28.945688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:28.945917 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:28.946060 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:28.946069 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:28.946884 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:28.946888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:28.947759 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:28.947771 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:28.948885 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:28.948890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:28.949010 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:28.949573 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:28.949591 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:28.951393 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:28.953461 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:28.953465 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:28.953467 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:28.953792 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:28.954094 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:28.954100 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:28.954105 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:28.954215 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:28.954714 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:28.954719 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:28.954860 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 14:41:28.954980 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 14:41:28.955163 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260808 14:41:28.956470 1397 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:41:28.957293 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.957026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:41805 (local address 127.0.0.1:44344)
0808 14:41:28.957166 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.957221 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:41805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 14:41:28.958063 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.957841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:41805 (local address 127.0.0.1:44346)
0808 14:41:28.957963 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.958009 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:41805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260808 14:41:28.958213 1397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:41805: Network error: Client connection negotiation failed: client connection to 127.19.144.124:41805: connect: Connection refused (error 111)
I20260808 14:41:28.972641 1474 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } attempt: 2
I20260808 14:41:28.973591 1477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.973283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:41805 (local address 127.0.0.1:44352)
0808 14:41:28.973442 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.973508 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:41805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 14:41:28.974363 1477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:28.974165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:41805 (local address 127.0.0.1:44358)
0808 14:41:28.974259 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:28.974310 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:41805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 14:41:28.974501 1474 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:41805: Network error: Client connection negotiation failed: client connection to 127.19.144.124:41805: connect: Connection refused (error 111)
W20260808 14:41:28.980245 1480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:28.980543 1480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:28.980620 1480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:28.985351 1480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:41:28.985426 1480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:28.985446 1480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:28.985463 1480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:41:28.985510 1480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:41:28.991287 1480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:39547,127.19.144.125:42683,127.19.144.124:41805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.124.keytab
--principal=kudu/127.19.144.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.124:41805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.144.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:28.992522 1480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:28.993738 1480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:29.003069 1486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:29.003108 1480 server_base.cc:1034] running on GCE node
W20260808 14:41:29.003077 1487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:29.003114 1489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:29.004106 1480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:29.005393 1480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:29.006592 1480 hybrid_clock.cc:648] HybridClock initialized: now 1786200089006574 us; error 61 us; skew 500 ppm
Aug 08 14:41:29 dist-test-slave-p3t5 krb5kdc[1314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200089, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:29.010380 1480 init.cc:377] Logged in from keytab as kudu/127.19.144.124@KRBTEST.COM (short username kudu)
I20260808 14:41:29.011693 1480 webserver.cc:460] Webserver started at http://127.19.144.124:35939/ using document root <none> and password file <none>
I20260808 14:41:29.012549 1480 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:29.012650 1480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:29.012933 1480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:41:29.015453 1480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/data/instance:
uuid: "e80ae67975ac49a999a469f4c48d1ac6"
format_stamp: "Formatted at 2026-08-08 14:41:29 on dist-test-slave-p3t5"
server_key: "dcd17efcf4e54ab2cf89903531b62080"
server_key_iv: "880fa79b8c6bb73486e9d4e088080738"
server_key_version: "encryptionkey@0"
I20260808 14:41:29.016113 1480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/wal/instance:
uuid: "e80ae67975ac49a999a469f4c48d1ac6"
format_stamp: "Formatted at 2026-08-08 14:41:29 on dist-test-slave-p3t5"
server_key: "dcd17efcf4e54ab2cf89903531b62080"
server_key_iv: "880fa79b8c6bb73486e9d4e088080738"
server_key_version: "encryptionkey@0"
I20260808 14:41:29.017391 1397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } attempt: 1
I20260808 14:41:29.018268 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.017965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:41805 (local address 127.0.0.1:44370)
0808 14:41:29.018115 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:29.018181 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:41805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 14:41:29.019079 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.018859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:41805 (local address 127.0.0.1:44374)
0808 14:41:29.018968 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:29.019018 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:41805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 14:41:29.019205 1397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:41805: Network error: Client connection negotiation failed: client connection to 127.19.144.124:41805: connect: Connection refused (error 111)
I20260808 14:41:29.020642 1480 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 14:41:29.023458 1496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:29.024834 1480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:41:29.025024 1480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/wal
uuid: "e80ae67975ac49a999a469f4c48d1ac6"
format_stamp: "Formatted at 2026-08-08 14:41:29 on dist-test-slave-p3t5"
server_key: "dcd17efcf4e54ab2cf89903531b62080"
server_key_iv: "880fa79b8c6bb73486e9d4e088080738"
server_key_version: "encryptionkey@0"
I20260808 14:41:29.025159 1480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:29.035785 1480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:29.039623 1480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:29.039857 1480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:29.047801 1480 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.124:41805
I20260808 14:41:29.048123 1549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.124:41805 every 8 connection(s)
I20260808 14:41:29.048914 1480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/data/info.pb
I20260808 14:41:29.049330 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 1480
I20260808 14:41:29.049489 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786199922348708-20033-0/minicluster-data/master-2/wal/instance
I20260808 14:41:29.049860 20033 external_mini_cluster.cc:1428] Setting key f6fb54d6decf6098e5a3ba1f1b9c0aaa
I20260808 14:41:29.052484 1550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:29.057153 1550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:41:29 dist-test-slave-p3t5 krb5kdc[1314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.124@KRBTEST.COM
Aug 08 14:41:29 dist-test-slave-p3t5 krb5kdc[1314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200089, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.124@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:29.065673 1474 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } attempt: 3
I20260808 14:41:29.066094 1553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.051437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44380 (local address 127.19.144.124:41805)
0808 14:41:29.051862 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:29.051879 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:29.051921 (+ 42us) server_negotiation.cc:407] Connection header received
0808 14:41:29.052737 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:29.052758 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:29.053193 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:29.053496 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:29.054530 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.055615 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.056834 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.057375 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.057516 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:29.060899 (+ 3383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:29.060905 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:29.060915 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:29.060946 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:29.062876 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.063398 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.063404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.063409 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.063544 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.063846 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.063850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.063852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.064286 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:29.064517 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:29.064787 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 14:41:29.065156 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":133,"threads_started":1}
I20260808 14:41:29.070292 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.059556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51280 (local address 127.19.144.126:39547)
0808 14:41:29.059721 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:29.059724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:29.060152 (+ 428us) server_negotiation.cc:407] Connection header received
0808 14:41:29.060432 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:29.060438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:29.060544 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:29.060609 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:29.061485 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.062058 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.062977 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.063195 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.063309 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:29.065525 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:29.065528 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:29.065530 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:29.065556 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:29.068169 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.068685 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.068689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.068691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.068798 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.069147 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.069150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.069152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.069429 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:29.069516 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:29.070033 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 14:41:29.070138 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 14:41:29.070947 1554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.059530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:39547 (local address 127.0.0.1:51280)
0808 14:41:29.060019 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:29.060065 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:29.060291 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:29.060833 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:29.060839 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:29.061217 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:29.061350 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:29.061357 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.062176 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.062179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:29.062847 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:29.062857 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.063313 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.063317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:29.063446 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:29.063963 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:29.063976 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:29.065371 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:29.068283 (+ 2912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:29.068294 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:29.068300 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:29.068572 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:29.068894 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:29.068900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:29.068906 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:29.069016 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:29.069603 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:29.069610 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:29.069774 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 14:41:29.070520 (+ 746us) client_negotiation.cc:240] Negotiation successful
0808 14:41:29.070718 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":216,"threads_started":1}
I20260808 14:41:29.072134 1550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:41:29 dist-test-slave-p3t5 krb5kdc[1314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200088, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.125@KRBTEST.COM for kudu/127.19.144.124@KRBTEST.COM
Aug 08 14:41:29 dist-test-slave-p3t5 krb5kdc[1314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:29.077935 1477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.066443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:41805 (local address 127.0.0.1:44388)
0808 14:41:29.066740 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:29.066758 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:29.066853 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:29.067767 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:29.067770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:29.067941 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:29.068031 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:29.068037 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.069071 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.069075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:29.069750 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:29.069763 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.070473 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.070477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:29.070566 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:29.071158 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:29.071174 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:29.072730 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:29.075390 (+ 2660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:29.075393 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:29.075396 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:29.075686 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:29.076618 (+ 932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:29.076622 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:29.076625 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:29.076723 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:29.077242 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:29.077245 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:29.077347 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 14:41:29.077485 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 14:41:29.077688 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201}
I20260808 14:41:29.078666 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.067204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44388 (local address 127.19.144.124:41805)
0808 14:41:29.067516 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:29.067525 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:29.067540 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:41:29.067575 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:29.067579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:29.067624 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:29.067739 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:29.068147 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.068924 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.069917 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.070321 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.070826 (+ 505us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:29.072842 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:29.072846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:29.072850 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:29.072878 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:29.075274 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.075791 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.075794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.075796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.075908 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.076839 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.076845 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.076848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.077110 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:29.077243 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:29.077769 (+ 526us) server_negotiation.cc:299] Negotiation successful
0808 14:41:29.077917 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":119,"threads_started":1}
Aug 08 14:41:29 dist-test-slave-p3t5 krb5kdc[1314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200089, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.124@KRBTEST.COM for kudu/127.19.144.125@KRBTEST.COM
I20260808 14:41:29.080044 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.069094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51292 (local address 127.19.144.126:39547)
0808 14:41:29.069609 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:29.069656 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:29.069669 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:41:29.069700 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:29.069703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:29.069748 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:29.069843 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:29.070688 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.071548 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.072779 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.073162 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.073656 (+ 494us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:29.076146 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:29.076150 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:29.076152 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:29.076181 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:29.078234 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.078691 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.078694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.078696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.078798 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.079124 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.079126 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.079127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.079305 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:29.079545 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:29.079685 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 14:41:29.079819 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":71,"threads_started":1}
I20260808 14:41:29.082754 1474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840: Bootstrap starting.
I20260808 14:41:29.083520 1554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.072664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:42683 (local address 127.0.0.1:53602)
0808 14:41:29.072932 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:29.072947 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:29.073039 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:29.073322 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:29.073328 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:29.073518 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:29.074090 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:29.074100 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.075168 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.075174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:29.075921 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:29.075929 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.076735 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.076739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:29.076891 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:29.077587 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:29.077601 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:29.079595 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:29.081812 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:29.081815 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:29.081818 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:29.082071 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:29.082385 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:29.082387 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:29.082388 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:29.082464 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:29.082979 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:29.082982 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:29.083085 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 14:41:29.083192 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 14:41:29.083354 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193}
I20260808 14:41:29.083519 1479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.072726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53602 (local address 127.19.144.125:42683)
0808 14:41:29.072841 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:29.072845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:29.073023 (+ 178us) server_negotiation.cc:407] Connection header received
0808 14:41:29.073135 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:29.073139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:29.073191 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:29.073277 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:29.074228 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.075038 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.076035 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.076378 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.076506 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:29.079748 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:29.079751 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:29.079754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:29.079779 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:29.081684 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.082181 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.082184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.082186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.082269 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.082557 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.082562 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.082564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.082772 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:29.082991 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:29.083249 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 14:41:29.083363 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 14:41:29.084625 1550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:41:29.086220 1474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:29.086635 1397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } attempt: 2
I20260808 14:41:29.087363 1474 log.cc:824] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:41:29 dist-test-slave-p3t5 krb5kdc[1314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.125@KRBTEST.COM
I20260808 14:41:29.090649 1474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840: No bootstrap required, opened a new log
Aug 08 14:41:29 dist-test-slave-p3t5 krb5kdc[1314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200089, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.124@KRBTEST.COM for kudu/127.19.144.124@KRBTEST.COM
Aug 08 14:41:29 dist-test-slave-p3t5 krb5kdc[1314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200088, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for kudu/127.19.144.124@KRBTEST.COM
I20260808 14:41:29.093988 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.081531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53614 (local address 127.19.144.125:42683)
0808 14:41:29.081937 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:29.081948 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:29.081960 (+ 12us) server_negotiation.cc:407] Connection header received
0808 14:41:29.081999 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:29.082004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:29.082049 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:29.082149 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:29.082957 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.083751 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.084599 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.084963 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.085088 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:29.089725 (+ 4637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:29.089728 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:29.089732 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:29.089760 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:29.091768 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.092364 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.092370 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.092374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.092474 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.092853 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.092856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.092858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.093114 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:29.093325 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:29.093576 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 14:41:29.093757 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":149,"threads_started":1}
I20260808 14:41:29.097173 1474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } }
I20260808 14:41:29.097508 1474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:29.097576 1474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f52265b04294161aa504efcacbca840, State: Initialized, Role: FOLLOWER
I20260808 14:41:29.098294 1554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.085177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:41805 (local address 127.0.0.1:44392)
0808 14:41:29.085300 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:29.085314 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:29.085506 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:29.088306 (+ 2800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:29.088309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:29.088488 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:29.088554 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:29.088559 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.089310 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.089316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:29.090010 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:29.090024 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.090725 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.090730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:29.090814 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:29.091710 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:29.091722 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:29.093926 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:29.096416 (+ 2490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:29.096420 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:29.096423 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:29.096750 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:29.097088 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:29.097092 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:29.097096 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:29.097183 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:29.097749 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:29.097754 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:29.097848 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 14:41:29.097953 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 14:41:29.098112 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 14:41:29.098335 1474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } }
I20260808 14:41:29.098773 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.085415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44392 (local address 127.19.144.124:41805)
0808 14:41:29.085580 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:29.085586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:29.085599 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:41:29.085661 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:29.085664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:29.088186 (+ 2522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:29.088294 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:29.088668 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.089193 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.090140 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.090490 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.090602 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:29.094084 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:29.094090 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:29.094094 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:29.094119 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:29.096297 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.096865 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.096869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.096871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.096980 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.097301 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.097303 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.097305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.097570 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:29.097719 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:29.098497 (+ 778us) server_negotiation.cc:299] Negotiation successful
0808 14:41:29.098621 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":53}
I20260808 14:41:29.100191 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.087151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:41805 (local address 127.0.0.1:44396)
0808 14:41:29.087322 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:29.087340 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:29.087427 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:29.088424 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:29.088428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:29.088658 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:29.088776 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:29.088784 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.090039 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.090043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:29.090954 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:29.090969 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.091605 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.091612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:29.091711 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:29.092204 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:29.092221 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:29.095181 (+ 2960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:29.097768 (+ 2587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:29.097772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:29.097775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:29.098098 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:29.098794 (+ 696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:29.098797 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:29.098799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:29.099124 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:29.099616 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:29.099619 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:29.099742 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 14:41:29.099872 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 14:41:29.100039 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 14:41:29.100200 1474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:41:29.100533 1553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.087193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44396 (local address 127.19.144.124:41805)
0808 14:41:29.087638 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:29.087642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:29.087656 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:41:29.087697 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:29.087699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:29.088219 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:29.088319 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:29.089042 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.089882 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.091087 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.091456 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.091579 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:29.095355 (+ 3776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:29.095359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:29.095362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:29.095393 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:29.097374 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.098236 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.098240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.098242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.098336 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.099254 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.099257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.099259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.099483 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:29.099598 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:29.099960 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 14:41:29.100096 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":593,"server-negotiator.queue_time_us":330,"spinlock_wait_cycles":1012096}
I20260808 14:41:29.102258 1550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6: Bootstrap starting.
I20260808 14:41:29.103551 1559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } } }
I20260808 14:41:29.103708 1559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:41:29.104159 1397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f: Bootstrap starting.
W20260808 14:41:29.106613 1570 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:41:29.106710 1570 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:41:29.107529 1397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:29.105151 1404 master_runner.cc:186] Error getting master registration for 127.19.144.126:39547: OK, instance_id { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" instance_seqno: 1786200088547463 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 14:41:29.108296 1404 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 14:41:29.108455 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:41:29.108654 1397 log.cc:824] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:29.111619 1397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f: No bootstrap required, opened a new log
I20260808 14:41:29.112479 1550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:29.113689 1550 log.cc:824] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:29.116060 1397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } }
I20260808 14:41:29.116302 1397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:29.116364 1397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e8df0db57a14443ae2d9ef82831829f, State: Initialized, Role: FOLLOWER
I20260808 14:41:29.116603 1550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6: No bootstrap required, opened a new log
I20260808 14:41:29.116901 1397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } }
I20260808 14:41:29.117439 1583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } } }
I20260808 14:41:29.117666 1583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:41:29.118637 1397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:41:29.121528 1550 raft_consensus.cc:348] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } }
I20260808 14:41:29.122046 1550 raft_consensus.cc:374] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:29.122118 1550 raft_consensus.cc:729] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e80ae67975ac49a999a469f4c48d1ac6, State: Initialized, Role: FOLLOWER
I20260808 14:41:29.122722 1550 consensus_queue.cc:260] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } }
I20260808 14:41:29.124361 1550 sys_catalog.cc:564] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:41:29.128937 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.111329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51308 (local address 127.19.144.126:39547)
0808 14:41:29.111511 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:29.111516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:29.117992 (+ 6476us) server_negotiation.cc:407] Connection header received
0808 14:41:29.118186 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:29.118190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:29.118250 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:29.118334 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:29.119522 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.120463 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.121867 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.122187 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.122271 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:29.124612 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:29.124615 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:29.124618 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:29.124647 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:29.126495 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.127075 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.127082 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.127087 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.127189 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.127770 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.127774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.127775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.128035 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:29.128148 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:29.128610 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 14:41:29.128756 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 14:41:29.130095 1327 master_runner.cc:263] No leader master found from master 1e8df0db57a14443ae2d9ef82831829f
I20260808 14:41:29.130167 1327 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 14:41:29.134320 1480 master_runner.cc:263] No leader master found from master e80ae67975ac49a999a469f4c48d1ac6
I20260808 14:41:29.134418 1480 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 14:41:29.135455 1596 sys_catalog.cc:455] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } } }
I20260808 14:41:29.135612 1596 sys_catalog.cc:458] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 14:41:29.137539 1595 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:41:29.137676 1595 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 14:41:29.138145 1607 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:41:29.138239 1607 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:41:29.142066 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.112333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53616 (local address 127.19.144.125:42683)
0808 14:41:29.112451 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:29.112456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:29.125709 (+ 13253us) server_negotiation.cc:407] Connection header received
0808 14:41:29.125965 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:29.125968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:29.126025 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:29.126094 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:29.132136 (+ 6042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.133014 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.134281 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.134674 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.134766 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:29.137579 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:29.137585 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:29.137588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:29.137651 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:29.139501 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.140158 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.140163 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.140165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.140267 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.140588 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.140592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.140594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.140862 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:29.141015 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:29.141728 (+ 713us) server_negotiation.cc:299] Negotiation successful
0808 14:41:29.141885 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:41:29.146782 1553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:29.113121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44402 (local address 127.19.144.124:41805)
0808 14:41:29.113252 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:29.113257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:29.127136 (+ 13879us) server_negotiation.cc:407] Connection header received
0808 14:41:29.127360 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:29.127366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:29.127433 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:29.127500 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:29.128538 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.129367 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.130700 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:29.131041 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:29.131127 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:29.142605 (+ 11478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:29.142610 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:29.142613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:29.142646 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:29.144577 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.145174 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.145179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.145181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.145280 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:29.145638 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:29.145642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:29.145644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:29.145904 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:29.146046 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:29.146439 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 14:41:29.146577 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:41:29.173976 1559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:41:29.174229 1559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } }
I20260808 14:41:29.175585 1559 leader_election.cc:288] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e8df0db57a14443ae2d9ef82831829f (127.19.144.126:39547), e80ae67975ac49a999a469f4c48d1ac6 (127.19.144.124:41805)
I20260808 14:41:29.176409 1370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0f52265b04294161aa504efcacbca840" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e8df0db57a14443ae2d9ef82831829f" is_pre_election: true
I20260808 14:41:29.176841 1370 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f52265b04294161aa504efcacbca840 in term 0.
I20260808 14:41:29.177381 1424 leader_election.cc:302] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f52265b04294161aa504efcacbca840, 1e8df0db57a14443ae2d9ef82831829f; no voters:
I20260808 14:41:29.178362 1523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0f52265b04294161aa504efcacbca840" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e80ae67975ac49a999a469f4c48d1ac6" is_pre_election: true
I20260808 14:41:29.178750 1523 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f52265b04294161aa504efcacbca840 in term 0.
I20260808 14:41:29.182479 1559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:41:29.182648 1559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:41:29.182717 1559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:29.184353 1559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } }
I20260808 14:41:29.185492 1370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0f52265b04294161aa504efcacbca840" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e8df0db57a14443ae2d9ef82831829f"
I20260808 14:41:29.185721 1370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:29.187374 1370 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f52265b04294161aa504efcacbca840 in term 1.
I20260808 14:41:29.187913 1424 leader_election.cc:302] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f52265b04294161aa504efcacbca840, 1e8df0db57a14443ae2d9ef82831829f; no voters:
I20260808 14:41:29.191973 1559 leader_election.cc:288] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [CANDIDATE]: Term 1 election: Requested vote from peers 1e8df0db57a14443ae2d9ef82831829f (127.19.144.126:39547), e80ae67975ac49a999a469f4c48d1ac6 (127.19.144.124:41805)
I20260808 14:41:29.192148 1559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:41:29.192250 1559 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [term 1 LEADER]: Becoming Leader. State: Replica: 0f52265b04294161aa504efcacbca840, State: Running, Role: LEADER
I20260808 14:41:29.192667 1559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } }
I20260808 14:41:29.194028 1523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0f52265b04294161aa504efcacbca840" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e80ae67975ac49a999a469f4c48d1ac6"
I20260808 14:41:29.194196 1523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:29.195716 1523 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f52265b04294161aa504efcacbca840 in term 1.
I20260808 14:41:29.197561 1559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f52265b04294161aa504efcacbca840. Latest consensus state: current_term: 1 leader_uuid: "0f52265b04294161aa504efcacbca840" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } } }
I20260808 14:41:29.197726 1559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [sys.catalog]: This master's current role is: LEADER
I20260808 14:41:29.198577 1612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:41:29.203349 1612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:41:29.212633 1523 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6 [term 1 FOLLOWER]: Refusing update from remote peer 0f52265b04294161aa504efcacbca840: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:41:29.216284 1610 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:41:29.221952 1596 sys_catalog.cc:455] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f52265b04294161aa504efcacbca840. Latest consensus state: current_term: 1 leader_uuid: "0f52265b04294161aa504efcacbca840" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } } }
I20260808 14:41:29.222165 1596 sys_catalog.cc:458] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:41:29.223685 1370 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f [term 1 FOLLOWER]: Refusing update from remote peer 0f52265b04294161aa504efcacbca840: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:41:29.224521 1610 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:41:29.232100 1583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f52265b04294161aa504efcacbca840. Latest consensus state: current_term: 1 leader_uuid: "0f52265b04294161aa504efcacbca840" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } } }
I20260808 14:41:29.232293 1583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:41:29.239429 1610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f52265b04294161aa504efcacbca840" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } } }
I20260808 14:41:29.239620 1610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [sys.catalog]: This master's current role is: LEADER
I20260808 14:41:29.241559 1583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f52265b04294161aa504efcacbca840" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } } }
I20260808 14:41:29.241734 1583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:41:29.242353 1613 mvcc.cc:205] Tried to move back new op lower bound from 7316275565404831744 to 7316275565347733504. Current Snapshot: MvccSnapshot[applied={T|T < 7316275565404831744}]
I20260808 14:41:29.245734 1616 mvcc.cc:205] Tried to move back new op lower bound from 7316275565404831744 to 7316275565347733504. Current Snapshot: MvccSnapshot[applied={T|T < 7316275565404831744}]
I20260808 14:41:29.250342 1559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f52265b04294161aa504efcacbca840" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } } }
I20260808 14:41:29.250509 1559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840 [sys.catalog]: This master's current role is: LEADER
I20260808 14:41:29.252029 1596 sys_catalog.cc:455] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f52265b04294161aa504efcacbca840" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e8df0db57a14443ae2d9ef82831829f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39547 } } peers { permanent_uuid: "0f52265b04294161aa504efcacbca840" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 42683 } } peers { permanent_uuid: "e80ae67975ac49a999a469f4c48d1ac6" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 41805 } } }
I20260808 14:41:29.252169 1596 sys_catalog.cc:458] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:41:29.259174 1614 mvcc.cc:205] Tried to move back new op lower bound from 7316275565404831744 to 7316275565347733504. Current Snapshot: MvccSnapshot[applied={T|T < 7316275565404831744}]
I20260808 14:41:29.260505 1612 catalog_manager.cc:1347] Generated new cluster ID: be5bbed7feed48b1abf690e860c9c18b
I20260808 14:41:29.260582 1612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:41:29.295496 1612 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:41:29.297065 1612 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:41:29.320978 1612 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f52265b04294161aa504efcacbca840: Generated new TSK 0
I20260808 14:41:29.322163 1612 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:41:30.139171 1595 catalog_manager.cc:1259] Loaded cluster ID: be5bbed7feed48b1abf690e860c9c18b
I20260808 14:41:30.139266 1595 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f: loading cluster ID for follower catalog manager: success
I20260808 14:41:30.139436 1607 catalog_manager.cc:1259] Loaded cluster ID: be5bbed7feed48b1abf690e860c9c18b
I20260808 14:41:30.139519 1607 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6: loading cluster ID for follower catalog manager: success
I20260808 14:41:30.141093 1607 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6: acquiring CA information for follower catalog manager: success
I20260808 14:41:30.141093 1595 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f: acquiring CA information for follower catalog manager: success
I20260808 14:41:30.142392 1595 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e8df0db57a14443ae2d9ef82831829f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 14:41:30.142392 1607 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e80ae67975ac49a999a469f4c48d1ac6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 14:41:30.393962 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 1327
I20260808 14:41:30.403707 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 1404
I20260808 14:41:30.411579 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 1480
2026-08-08T14:41:30Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3366 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:41:30 dist-test-slave-p3t5 krb5kdc[1627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:41:30 dist-test-slave-p3t5 krb5kdc[1627](info): set up 2 sockets
Aug 08 14:41:30 dist-test-slave-p3t5 krb5kdc[1627](info): commencing operation
krb5kdc: starting...
W20260808 14:41:31.487699 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:41:31 dist-test-slave-p3t5 krb5kdc[1627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:41:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:41:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:41:31.628566 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:43819
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:43819
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:41:31.752645 1640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:31.753031 1640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:31.753116 1640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:31.757532 1640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:41:31.757668 1640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:31.757709 1640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:31.757776 1640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:41:31.757817 1640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:41:31.763751 1640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:43819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:43819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:31.765085 1640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:31.766317 1640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:31.776221 1645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:31.776233 1648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:31.776233 1646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:31.776656 1640 server_base.cc:1034] running on GCE node
I20260808 14:41:31.777439 1640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:31.778790 1640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:31.780004 1640 hybrid_clock.cc:648] HybridClock initialized: now 1786200091779985 us; error 39 us; skew 500 ppm
Aug 08 14:41:31 dist-test-slave-p3t5 krb5kdc[1627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200091, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:31.783314 1640 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:41:31.784400 1640 webserver.cc:460] Webserver started at http://127.19.144.126:42959/ using document root <none> and password file <none>
I20260808 14:41:31.785039 1640 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:31.785135 1640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:31.785383 1640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:41:31.787777 1640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "7a5c500e08b24a74baf74ac8c5fc813f"
format_stamp: "Formatted at 2026-08-08 14:41:31 on dist-test-slave-p3t5"
server_key: "f303778a2e3302ebf1ca9d2c08f4ae6e"
server_key_iv: "9f9e9143ac327292770212ad4a78f600"
server_key_version: "encryptionkey@0"
I20260808 14:41:31.788442 1640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "7a5c500e08b24a74baf74ac8c5fc813f"
format_stamp: "Formatted at 2026-08-08 14:41:31 on dist-test-slave-p3t5"
server_key: "f303778a2e3302ebf1ca9d2c08f4ae6e"
server_key_iv: "9f9e9143ac327292770212ad4a78f600"
server_key_version: "encryptionkey@0"
I20260808 14:41:31.792743 1640 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:41:31.795723 1655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:31.797205 1640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:41:31.797377 1640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "7a5c500e08b24a74baf74ac8c5fc813f"
format_stamp: "Formatted at 2026-08-08 14:41:31 on dist-test-slave-p3t5"
server_key: "f303778a2e3302ebf1ca9d2c08f4ae6e"
server_key_iv: "9f9e9143ac327292770212ad4a78f600"
server_key_version: "encryptionkey@0"
I20260808 14:41:31.797526 1640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:31.813827 1640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:31.817479 1640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:31.817739 1640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:31.825326 1640 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:43819
I20260808 14:41:31.825345 1708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:43819 every 8 connection(s)
I20260808 14:41:31.826417 1640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:41:31.827133 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 1640
I20260808 14:41:31.827271 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:41:31.827569 20033 external_mini_cluster.cc:1428] Setting key d9295da0041928c1dbe0b70622de8444
I20260808 14:41:31.829744 1709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:31.835669 1709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f: Bootstrap starting.
Aug 08 14:41:31 dist-test-slave-p3t5 krb5kdc[1627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:31.838444 1709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:31.839324 1709 log.cc:824] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:31.841876 1709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f: No bootstrap required, opened a new log
I20260808 14:41:31.843361 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:31.828957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47384 (local address 127.19.144.126:43819)
0808 14:41:31.829435 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:31.829458 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:31.829782 (+ 324us) server_negotiation.cc:407] Connection header received
0808 14:41:31.830587 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:31.830605 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:31.830967 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:31.831332 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:31.832179 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:31.833215 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:31.834212 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:31.834772 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:31.834956 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:31.837939 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:31.837944 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:31.837959 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:31.837990 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:31.839981 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:31.840608 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:31.840613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:31.840618 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:31.840738 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:31.841076 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:31.841080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:31.841081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:31.841544 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:31.841797 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:31.842196 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 14:41:31.842550 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":171,"threads_started":1}
I20260808 14:41:31.845860 1709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c500e08b24a74baf74ac8c5fc813f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43819 } }
I20260808 14:41:31.846107 1709 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:31.846181 1709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a5c500e08b24a74baf74ac8c5fc813f, State: Initialized, Role: FOLLOWER
I20260808 14:41:31.846484 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:41:31.846688 1709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c500e08b24a74baf74ac8c5fc813f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43819 } }
I20260808 14:41:31.846865 1709 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:41:31.846932 1709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:41:31.847056 1709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:31.848074 1709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c500e08b24a74baf74ac8c5fc813f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43819 } }
I20260808 14:41:31.848455 1709 leader_election.cc:302] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a5c500e08b24a74baf74ac8c5fc813f; no voters:
I20260808 14:41:31.848762 1709 leader_election.cc:288] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:41:31.848896 1714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:41:31.849165 1714 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [term 1 LEADER]: Becoming Leader. State: Replica: 7a5c500e08b24a74baf74ac8c5fc813f, State: Running, Role: LEADER
I20260808 14:41:31.849531 1714 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c500e08b24a74baf74ac8c5fc813f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43819 } }
I20260808 14:41:31.850069 1709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:41:31.851329 1716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a5c500e08b24a74baf74ac8c5fc813f. Latest consensus state: current_term: 1 leader_uuid: "7a5c500e08b24a74baf74ac8c5fc813f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c500e08b24a74baf74ac8c5fc813f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43819 } } }
I20260808 14:41:31.851468 1716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [sys.catalog]: This master's current role is: LEADER
I20260808 14:41:31.851886 1723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:41:31.851859 1715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a5c500e08b24a74baf74ac8c5fc813f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5c500e08b24a74baf74ac8c5fc813f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43819 } } }
I20260808 14:41:31.852109 1715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f [sys.catalog]: This master's current role is: LEADER
I20260808 14:41:31.854856 1723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:41:31.860289 1723 catalog_manager.cc:1347] Generated new cluster ID: 0ce23ec2ae5841fcb95245f76376aa3e
I20260808 14:41:31.860368 1723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:41:31.875075 1723 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:41:31.876544 1723 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:41:31.885905 1723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a5c500e08b24a74baf74ac8c5fc813f: Generated new TSK 0
I20260808 14:41:31.886830 1723 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:41:31.903237 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 1640
2026-08-08T14:41:31Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1490 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:41:31 dist-test-slave-p3t5 krb5kdc[1737](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:41:31 dist-test-slave-p3t5 krb5kdc[1737](info): set up 2 sockets
Aug 08 14:41:31 dist-test-slave-p3t5 krb5kdc[1737](info): commencing operation
krb5kdc: starting...
W20260808 14:41:32.979797 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:41:33 dist-test-slave-p3t5 krb5kdc[1737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:41:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:41:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:41:33.144718 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:43211
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:43211
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:41:33.264683 1751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:33.264991 1751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:33.265034 1751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:33.269389 1751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:41:33.269479 1751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:33.269498 1751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:33.269515 1751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:41:33.269531 1751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:41:33.275257 1751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:43211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:43211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:33.276428 1751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:33.277506 1751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:33.286427 1757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:33.286448 1756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:33.286448 1759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:33.286857 1751 server_base.cc:1034] running on GCE node
I20260808 14:41:33.287556 1751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:33.288841 1751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:33.290037 1751 hybrid_clock.cc:648] HybridClock initialized: now 1786200093290022 us; error 33 us; skew 500 ppm
Aug 08 14:41:33 dist-test-slave-p3t5 krb5kdc[1737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200093, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:33.293020 1751 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:41:33.294148 1751 webserver.cc:460] Webserver started at http://127.19.144.126:34367/ using document root <none> and password file <none>
I20260808 14:41:33.294723 1751 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:33.294790 1751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:33.294965 1751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:41:33.297123 1751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "bf92c5ea5ec240578d9e6f8f4b227c26"
format_stamp: "Formatted at 2026-08-08 14:41:33 on dist-test-slave-p3t5"
server_key: "a4a80bb651ede6c22dc4f6e53d29a87f"
server_key_iv: "1cff0132a8395ea3a72b3e7b2d4828eb"
server_key_version: "encryptionkey@0"
I20260808 14:41:33.297781 1751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "bf92c5ea5ec240578d9e6f8f4b227c26"
format_stamp: "Formatted at 2026-08-08 14:41:33 on dist-test-slave-p3t5"
server_key: "a4a80bb651ede6c22dc4f6e53d29a87f"
server_key_iv: "1cff0132a8395ea3a72b3e7b2d4828eb"
server_key_version: "encryptionkey@0"
I20260808 14:41:33.301801 1751 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:41:33.304471 1766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:33.305903 1751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:41:33.306044 1751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "bf92c5ea5ec240578d9e6f8f4b227c26"
format_stamp: "Formatted at 2026-08-08 14:41:33 on dist-test-slave-p3t5"
server_key: "a4a80bb651ede6c22dc4f6e53d29a87f"
server_key_iv: "1cff0132a8395ea3a72b3e7b2d4828eb"
server_key_version: "encryptionkey@0"
I20260808 14:41:33.306154 1751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:33.316039 1751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:33.319352 1751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:33.319585 1751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:33.328008 1751 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:43211
I20260808 14:41:33.328325 1818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:43211 every 8 connection(s)
I20260808 14:41:33.329195 1751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:41:33.332234 1819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:33.332831 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 1751
I20260808 14:41:33.333002 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:41:33.333298 20033 external_mini_cluster.cc:1428] Setting key 8e82219c7bc7cce807eedccf17038255
I20260808 14:41:33.338702 1819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: Bootstrap starting.
I20260808 14:41:33.341537 1819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:33.342552 1819 log.cc:824] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:41:33 dist-test-slave-p3t5 krb5kdc[1737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:33.344866 1819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: No bootstrap required, opened a new log
I20260808 14:41:33.348618 1822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:33.334908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43378 (local address 127.19.144.126:43211)
0808 14:41:33.335374 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:33.335390 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:33.335470 (+ 80us) server_negotiation.cc:407] Connection header received
0808 14:41:33.336180 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:33.336197 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:33.336551 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:33.336845 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:33.337796 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:33.338850 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:33.340015 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:33.340500 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:33.340632 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:33.343643 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:33.343649 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:33.343664 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:33.343691 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:33.345539 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:33.346053 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:33.346058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:33.346063 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:33.346265 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:33.346591 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:33.346594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:33.346595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:33.347023 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:33.347184 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:33.347516 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 14:41:33.347849 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":138,"threads_started":1}
I20260808 14:41:33.349066 1819 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } }
I20260808 14:41:33.349350 1819 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:33.349421 1819 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf92c5ea5ec240578d9e6f8f4b227c26, State: Initialized, Role: FOLLOWER
I20260808 14:41:33.349916 1819 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } }
I20260808 14:41:33.350049 1819 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:41:33.350131 1819 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:41:33.350246 1819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:33.351311 1819 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } }
I20260808 14:41:33.351667 1819 leader_election.cc:302] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf92c5ea5ec240578d9e6f8f4b227c26; no voters:
I20260808 14:41:33.351981 1819 leader_election.cc:288] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:41:33.352145 1824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:41:33.352442 1824 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 1 LEADER]: Becoming Leader. State: Replica: bf92c5ea5ec240578d9e6f8f4b227c26, State: Running, Role: LEADER
I20260808 14:41:33.352803 1824 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } }
I20260808 14:41:33.353303 1819 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:41:33.354708 1825 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } } }
I20260808 14:41:33.354851 1825 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [sys.catalog]: This master's current role is: LEADER
I20260808 14:41:33.355247 1826 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf92c5ea5ec240578d9e6f8f4b227c26. Latest consensus state: current_term: 1 leader_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } } }
I20260808 14:41:33.355382 1826 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [sys.catalog]: This master's current role is: LEADER
W20260808 14:41:33.358575 1839 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:41:33.358673 1839 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:41:33.358750 1834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:41:33.359623 1834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:41:33.366074 1834 catalog_manager.cc:1347] Generated new cluster ID: b9c9317e9cd846149f0952d7300db951
I20260808 14:41:33.366173 1834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:41:33.375598 1834 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:41:33.376902 1834 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:41:33.387658 1834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: Generated new TSK 0
I20260808 14:41:33.388588 1834 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:41:33.424919 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:43211
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:41:33.546325 1845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:33.546643 1845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:33.546733 1845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:33.550956 1845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:33.551034 1845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:33.551141 1845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:41:33.556972 1845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:43211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:33.558296 1845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:33.559446 1845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:33.569952 1851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:33.569955 1850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:33.569954 1853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:33.570154 1845 server_base.cc:1034] running on GCE node
I20260808 14:41:33.570725 1845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:33.571445 1845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:33.572619 1845 hybrid_clock.cc:648] HybridClock initialized: now 1786200093572589 us; error 51 us; skew 500 ppm
Aug 08 14:41:33 dist-test-slave-p3t5 krb5kdc[1737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200093, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:33.575532 1845 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:41:33.576519 1845 webserver.cc:460] Webserver started at http://127.19.144.65:36391/ using document root <none> and password file <none>
I20260808 14:41:33.577129 1845 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:33.577225 1845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:33.577471 1845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:41:33.579697 1845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "e383d29721074c8db3ba1c68ac20d57c"
format_stamp: "Formatted at 2026-08-08 14:41:33 on dist-test-slave-p3t5"
server_key: "2e1e830b9634ec987cd2a84d56900712"
server_key_iv: "33c17947ca736e24b03300a8d816ce58"
server_key_version: "encryptionkey@0"
I20260808 14:41:33.580334 1845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "e383d29721074c8db3ba1c68ac20d57c"
format_stamp: "Formatted at 2026-08-08 14:41:33 on dist-test-slave-p3t5"
server_key: "2e1e830b9634ec987cd2a84d56900712"
server_key_iv: "33c17947ca736e24b03300a8d816ce58"
server_key_version: "encryptionkey@0"
I20260808 14:41:33.584630 1845 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:41:33.587390 1860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:33.588696 1845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:41:33.588867 1845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "e383d29721074c8db3ba1c68ac20d57c"
format_stamp: "Formatted at 2026-08-08 14:41:33 on dist-test-slave-p3t5"
server_key: "2e1e830b9634ec987cd2a84d56900712"
server_key_iv: "33c17947ca736e24b03300a8d816ce58"
server_key_version: "encryptionkey@0"
I20260808 14:41:33.589004 1845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:33.617885 1845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:33.621028 1845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:33.621255 1845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:33.621948 1845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:33.622866 1845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:41:33.622941 1845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:33.623015 1845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:41:33.623057 1845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:33.632261 1845 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:36149
I20260808 14:41:33.632288 1977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:36149 every 8 connection(s)
I20260808 14:41:33.633292 1845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
Aug 08 14:41:33 dist-test-slave-p3t5 krb5kdc[1737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200093, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:33.643605 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 1845
I20260808 14:41:33.643757 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:41:33.644047 20033 external_mini_cluster.cc:1428] Setting key 0434a921bc1ec6b256f882677cba2d38
I20260808 14:41:33.648416 1822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:33.635283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:36921 (local address 127.19.144.126:43211)
0808 14:41:33.635440 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:33.635444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:33.636383 (+ 939us) server_negotiation.cc:407] Connection header received
0808 14:41:33.637828 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:33.637832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:33.637884 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:33.637964 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:33.639223 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:33.639829 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:33.640986 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:33.641233 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:33.641331 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:33.643986 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:33.643990 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:33.644006 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:33.644035 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:33.645921 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:33.646515 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:33.646521 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:33.646525 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:33.646627 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:33.647019 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:33.647026 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:33.647030 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:33.647308 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:33.647416 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:33.648084 (+ 668us) server_negotiation.cc:299] Negotiation successful
0808 14:41:33.648226 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 14:41:33.649324 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:33.635696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:36921)
0808 14:41:33.636219 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:33.636267 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:33.637587 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:33.638142 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:33.638149 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:33.638559 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:33.639064 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:33.639078 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:33.639946 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:33.639949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:33.640866 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:33.640876 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:33.641382 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:33.641385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:33.641542 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:33.642160 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:33.642177 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:33.643787 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:33.646080 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:33.646086 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:33.646101 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:33.646404 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:33.646769 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:33.646772 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:33.646773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:33.646925 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:33.647430 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:33.647437 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:33.647726 (+ 289us) client_negotiation.cc:769] Sending connection context
0808 14:41:33.648019 (+ 293us) client_negotiation.cc:240] Negotiation successful
0808 14:41:33.648402 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":164,"threads_started":1}
I20260808 14:41:33.650673 1978 heartbeater.cc:347] Connected to a master server at 127.19.144.126:43211
I20260808 14:41:33.650981 1978 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:33.651664 1978 heartbeater.cc:511] Master 127.19.144.126:43211 requested a full tablet report, sending...
I20260808 14:41:33.653518 1783 ts_manager.cc:194] Registered new tserver with Master: e383d29721074c8db3ba1c68ac20d57c (127.19.144.65:36149)
I20260808 14:41:33.654806 1783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:36921
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:41:33.676812 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:43211
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:41:33.798409 1983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:33.798722 1983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:33.798779 1983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:33.802838 1983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:33.802898 1983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:33.802959 1983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:41:33.809054 1983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:43211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:33.810492 1983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:33.811663 1983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:33.822402 1989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:33.822413 1991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:33.822465 1988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:33.822765 1983 server_base.cc:1034] running on GCE node
I20260808 14:41:33.823243 1983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:33.823915 1983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:33.825096 1983 hybrid_clock.cc:648] HybridClock initialized: now 1786200093825083 us; error 37 us; skew 500 ppm
Aug 08 14:41:33 dist-test-slave-p3t5 krb5kdc[1737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200093, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:33.828159 1983 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:41:33.829097 1983 webserver.cc:460] Webserver started at http://127.19.144.66:45029/ using document root <none> and password file <none>
I20260808 14:41:33.829788 1983 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:33.829890 1983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:33.830147 1983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:41:33.832722 1983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "b8352d4aa8bc4e8fba90de4fd54ae607"
format_stamp: "Formatted at 2026-08-08 14:41:33 on dist-test-slave-p3t5"
server_key: "b9da2594f4341d8a39eaf6f317e9588c"
server_key_iv: "53ffacb03f8f93be1e604fd432559d79"
server_key_version: "encryptionkey@0"
I20260808 14:41:33.833428 1983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "b8352d4aa8bc4e8fba90de4fd54ae607"
format_stamp: "Formatted at 2026-08-08 14:41:33 on dist-test-slave-p3t5"
server_key: "b9da2594f4341d8a39eaf6f317e9588c"
server_key_iv: "53ffacb03f8f93be1e604fd432559d79"
server_key_version: "encryptionkey@0"
I20260808 14:41:33.837796 1983 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:41:33.840607 1998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:33.842125 1983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:41:33.842276 1983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "b8352d4aa8bc4e8fba90de4fd54ae607"
format_stamp: "Formatted at 2026-08-08 14:41:33 on dist-test-slave-p3t5"
server_key: "b9da2594f4341d8a39eaf6f317e9588c"
server_key_iv: "53ffacb03f8f93be1e604fd432559d79"
server_key_version: "encryptionkey@0"
I20260808 14:41:33.842410 1983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:33.869464 1983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:33.872925 1983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:33.873179 1983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:33.873962 1983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:33.874890 1983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:41:33.874977 1983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:33.875053 1983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:41:33.875103 1983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:33.885033 1983 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:35605
I20260808 14:41:33.885059 2113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:35605 every 8 connection(s)
I20260808 14:41:33.886101 1983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:41:33.895640 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 1983
I20260808 14:41:33.895792 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:41:33.896123 20033 external_mini_cluster.cc:1428] Setting key 93f00fbede1e37a013c0dcd93dc372a6
Aug 08 14:41:33 dist-test-slave-p3t5 krb5kdc[1737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200093, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:33.901484 1822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:33.888307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:51449 (local address 127.19.144.126:43211)
0808 14:41:33.888458 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:33.888462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:33.889465 (+ 1003us) server_negotiation.cc:407] Connection header received
0808 14:41:33.891011 (+ 1546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:33.891014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:33.891067 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:33.891155 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:33.892377 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:33.893031 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:33.894340 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:33.894586 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:33.894677 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:33.897682 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:33.897686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:33.897688 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:33.897715 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:33.899273 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:33.899846 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:33.899850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:33.899851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:33.899939 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:33.900312 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:33.900319 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:33.900322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:33.900525 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:33.900640 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:33.901175 (+ 535us) server_negotiation.cc:299] Negotiation successful
0808 14:41:33.901313 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 14:41:33.902452 2116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:33.888755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:51449)
0808 14:41:33.889250 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:33.889295 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:33.890765 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:33.891346 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:33.891352 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:33.891723 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:33.892214 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:33.892228 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:33.893164 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:33.893167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:33.894201 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:33.894212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:33.894731 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:33.894734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:33.894881 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:33.895558 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:33.895581 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:33.897431 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:33.899407 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:33.899414 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:33.899432 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:33.899730 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:33.900039 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:33.900043 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:33.900045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:33.900208 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:33.900638 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:33.900645 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:33.900924 (+ 279us) client_negotiation.cc:769] Sending connection context
0808 14:41:33.901131 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 14:41:33.901511 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":123,"threads_started":1}
I20260808 14:41:33.903934 2114 heartbeater.cc:347] Connected to a master server at 127.19.144.126:43211
I20260808 14:41:33.904227 2114 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:33.904830 2114 heartbeater.cc:511] Master 127.19.144.126:43211 requested a full tablet report, sending...
I20260808 14:41:33.906088 1783 ts_manager.cc:194] Registered new tserver with Master: b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605)
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
I20260808 14:41:33.907020 1783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:51449
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:41:33.929352 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:43211
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:41:34.054559 2119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:34.054862 2119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:34.054930 2119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:34.059248 2119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:34.059324 2119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:34.059406 2119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:41:34.065121 2119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:43211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:34.066354 2119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:34.067535 2119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:34.077906 2125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:34.078059 2119 server_base.cc:1034] running on GCE node
W20260808 14:41:34.077965 2127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:34.077979 2124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:34.078680 2119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:34.079345 2119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:34.080513 2119 hybrid_clock.cc:648] HybridClock initialized: now 1786200094080494 us; error 29 us; skew 500 ppm
Aug 08 14:41:34 dist-test-slave-p3t5 krb5kdc[1737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200094, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:34.083506 2119 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:41:34.084425 2119 webserver.cc:460] Webserver started at http://127.19.144.67:38335/ using document root <none> and password file <none>
I20260808 14:41:34.085033 2119 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:34.085127 2119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:34.085376 2119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:41:34.087795 2119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "7b934e85d4ac40f2b6adf6c68526024c"
format_stamp: "Formatted at 2026-08-08 14:41:34 on dist-test-slave-p3t5"
server_key: "5ceedeae0ac68b7836644c83bec9cc59"
server_key_iv: "3e0592cf1e3b2427c3fd5c449aac45a9"
server_key_version: "encryptionkey@0"
I20260808 14:41:34.088413 2119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "7b934e85d4ac40f2b6adf6c68526024c"
format_stamp: "Formatted at 2026-08-08 14:41:34 on dist-test-slave-p3t5"
server_key: "5ceedeae0ac68b7836644c83bec9cc59"
server_key_iv: "3e0592cf1e3b2427c3fd5c449aac45a9"
server_key_version: "encryptionkey@0"
I20260808 14:41:34.092662 2119 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 14:41:34.095526 2134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:34.097008 2119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:41:34.097172 2119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "7b934e85d4ac40f2b6adf6c68526024c"
format_stamp: "Formatted at 2026-08-08 14:41:34 on dist-test-slave-p3t5"
server_key: "5ceedeae0ac68b7836644c83bec9cc59"
server_key_iv: "3e0592cf1e3b2427c3fd5c449aac45a9"
server_key_version: "encryptionkey@0"
I20260808 14:41:34.097306 2119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:34.111685 2119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:34.115250 2119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:34.115509 2119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:34.116189 2119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:34.117105 2119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:41:34.117175 2119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:34.117249 2119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:41:34.117295 2119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:34.126569 2119 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:39659
I20260808 14:41:34.126638 2250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:39659 every 8 connection(s)
I20260808 14:41:34.127619 2119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:41:34.128093 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 2119
I20260808 14:41:34.128261 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:41:34.128597 20033 external_mini_cluster.cc:1428] Setting key 76c4f48420eca1521c4e66a994e3e673
Aug 08 14:41:34 dist-test-slave-p3t5 krb5kdc[1737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200094, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:34.143224 1822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.130039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:39137 (local address 127.19.144.126:43211)
0808 14:41:34.130189 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:34.130193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:34.131212 (+ 1019us) server_negotiation.cc:407] Connection header received
0808 14:41:34.132588 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:34.132590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:34.132635 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:34.132709 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:34.133986 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.134641 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.135810 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.136054 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.136141 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.138947 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:34.138951 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:34.138953 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:34.139000 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:34.140663 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:34.141279 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:34.141283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:34.141285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:34.141396 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:34.141889 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:34.141893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:34.141894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:34.142157 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:34.142266 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:34.142894 (+ 628us) server_negotiation.cc:299] Negotiation successful
0808 14:41:34.143040 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:41:34.144119 2253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.130540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:39137)
0808 14:41:34.131072 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:34.131118 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:34.132365 (+ 1247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:34.132846 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:34.132852 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:34.133231 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:34.133839 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.133857 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.134762 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.134764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.135690 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.135699 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.136226 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.136229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.136384 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.137061 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:34.137087 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:34.138743 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:34.140804 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:34.140811 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:34.140829 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:34.141157 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:34.141530 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:34.141533 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:34.141535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:34.141766 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:34.142271 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:34.142277 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:34.142631 (+ 354us) client_negotiation.cc:769] Sending connection context
0808 14:41:34.142796 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 14:41:34.143221 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":151,"threads_started":1}
I20260808 14:41:34.145226 2251 heartbeater.cc:347] Connected to a master server at 127.19.144.126:43211
I20260808 14:41:34.145511 2251 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:34.146143 2251 heartbeater.cc:511] Master 127.19.144.126:43211 requested a full tablet report, sending...
I20260808 14:41:34.147272 1783 ts_manager.cc:194] Registered new tserver with Master: 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67:39659)
I20260808 14:41:34.147856 1783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:39137
I20260808 14:41:34.155339 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:41:34.165985 1822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.157652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43392 (local address 127.19.144.126:43211)
0808 14:41:34.157811 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:34.157816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:34.158030 (+ 214us) server_negotiation.cc:407] Connection header received
0808 14:41:34.158220 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:34.158229 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:34.158273 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:34.158362 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:34.159139 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.159684 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.160588 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.160819 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.160936 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.162636 (+ 1700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:34.162640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:34.162642 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:34.162669 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:34.164081 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:34.164556 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:34.164560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:34.164562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:34.164658 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:34.164966 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:34.164970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:34.164972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:34.165235 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:34.165393 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:34.165672 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 14:41:34.165802 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 14:41:34.169286 1783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:41:34.172494 1783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:41:34.188334 2262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.181573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:35605 (local address 127.0.0.1:54510)
0808 14:41:34.182280 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:34.182325 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:34.182482 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:34.183603 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:34.183613 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:34.183643 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:34.183913 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.183922 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.185067 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.185071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.186479 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.186491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.187631 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.187635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.187786 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.187836 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 14:41:34.187989 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 14:41:34.188162 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":120,"thread_start_us":130,"threads_started":1}
I20260808 14:41:34.188340 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.181794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54510 (local address 127.19.144.66:35605)
0808 14:41:34.182986 (+ 1192us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:34.183000 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:34.183028 (+ 28us) server_negotiation.cc:407] Connection header received
0808 14:41:34.183117 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:34.183123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:34.183372 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:34.183590 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:34.184048 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.184917 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.186627 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.187482 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.187616 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.187794 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:34.188026 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 14:41:34.188146 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1063,"thread_start_us":152,"threads_started":1}
I20260808 14:41:34.188364 2264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.182011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:39659 (local address 127.0.0.1:54372)
0808 14:41:34.182594 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:34.182610 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:34.182702 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:34.183714 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:34.183717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:34.183731 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:34.183936 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.183947 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.185451 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.185454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.186664 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.186672 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.187729 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.187733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.187867 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.187896 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 14:41:34.187954 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 14:41:34.188179 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":111,"thread_start_us":110,"threads_started":1}
I20260808 14:41:34.188714 2265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.182421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:36149 (local address 127.0.0.1:40908)
0808 14:41:34.182899 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:34.182911 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:34.182990 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:34.183686 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:34.183689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:34.183718 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:34.184091 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.184099 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.185641 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.185645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.186970 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.186982 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.188314 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.188317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.188425 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.188449 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:41:34.188521 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 14:41:34.188583 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":171,"spinlock_wait_cycles":2688,"thread_start_us":99,"threads_started":1}
I20260808 14:41:34.189118 2267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.182724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40908 (local address 127.19.144.65:36149)
0808 14:41:34.183236 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:34.183243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:34.183265 (+ 22us) server_negotiation.cc:407] Connection header received
0808 14:41:34.183336 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:34.183354 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:34.183540 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:34.183850 (+ 310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:34.184217 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.185420 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.187140 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.187990 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.188100 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.188251 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:34.188843 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 14:41:34.188968 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":131,"threads_started":1}
I20260808 14:41:34.189435 2266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.182671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54372 (local address 127.19.144.67:39659)
0808 14:41:34.183165 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:34.183181 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:34.183208 (+ 27us) server_negotiation.cc:407] Connection header received
0808 14:41:34.183294 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:34.183301 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:34.183555 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:34.183903 (+ 348us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:34.184075 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.185294 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.186808 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.187577 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.188964 (+ 1387us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.189120 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:34.189178 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 14:41:34.189289 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":151,"threads_started":1}
I20260808 14:41:34.192216 1910 tablet_service.cc:1467] Processing CreateTablet for tablet bf58191b9d71451abcf9d7981379f3b4 (DEFAULT_TABLE table=test-table [id=eede6667c3db45359499406907cac6b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:41:34.192220 2046 tablet_service.cc:1467] Processing CreateTablet for tablet bf58191b9d71451abcf9d7981379f3b4 (DEFAULT_TABLE table=test-table [id=eede6667c3db45359499406907cac6b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:41:34.192217 2183 tablet_service.cc:1467] Processing CreateTablet for tablet bf58191b9d71451abcf9d7981379f3b4 (DEFAULT_TABLE table=test-table [id=eede6667c3db45359499406907cac6b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:41:34.193403 1910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf58191b9d71451abcf9d7981379f3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:34.193405 2183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf58191b9d71451abcf9d7981379f3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:34.193403 2046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf58191b9d71451abcf9d7981379f3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:34.201150 2268 tablet_bootstrap.cc:492] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c: Bootstrap starting.
I20260808 14:41:34.201817 2271 tablet_bootstrap.cc:492] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607: Bootstrap starting.
I20260808 14:41:34.201910 2270 tablet_bootstrap.cc:492] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c: Bootstrap starting.
I20260808 14:41:34.203806 2268 tablet_bootstrap.cc:654] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:34.204217 2271 tablet_bootstrap.cc:654] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:34.204421 2270 tablet_bootstrap.cc:654] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:34.204864 2268 log.cc:824] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:34.205014 2271 log.cc:824] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:34.205245 2270 log.cc:824] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:34.207322 2268 tablet_bootstrap.cc:492] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c: No bootstrap required, opened a new log
I20260808 14:41:34.207322 2271 tablet_bootstrap.cc:492] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607: No bootstrap required, opened a new log
I20260808 14:41:34.207512 2270 tablet_bootstrap.cc:492] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c: No bootstrap required, opened a new log
I20260808 14:41:34.207545 2271 ts_tablet_manager.cc:1397] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:41:34.207545 2268 ts_tablet_manager.cc:1397] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 14:41:34.207734 2270 ts_tablet_manager.cc:1397] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:41:34.211019 2270 raft_consensus.cc:348] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } }
I20260808 14:41:34.211295 2270 raft_consensus.cc:374] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:34.211352 2270 raft_consensus.cc:729] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b934e85d4ac40f2b6adf6c68526024c, State: Initialized, Role: FOLLOWER
I20260808 14:41:34.211843 2270 consensus_queue.cc:260] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } }
I20260808 14:41:34.211772 2271 raft_consensus.cc:348] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } }
I20260808 14:41:34.212050 2271 raft_consensus.cc:374] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:34.212103 2271 raft_consensus.cc:729] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8352d4aa8bc4e8fba90de4fd54ae607, State: Initialized, Role: FOLLOWER
I20260808 14:41:34.212319 2268 raft_consensus.cc:348] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } }
I20260808 14:41:34.212575 2268 raft_consensus.cc:374] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:34.212628 2268 raft_consensus.cc:729] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e383d29721074c8db3ba1c68ac20d57c, State: Initialized, Role: FOLLOWER
I20260808 14:41:34.212632 2251 heartbeater.cc:503] Master 127.19.144.126:43211 was elected leader, sending a full tablet report...
I20260808 14:41:34.212605 2271 consensus_queue.cc:260] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } }
I20260808 14:41:34.213071 2268 consensus_queue.cc:260] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } }
I20260808 14:41:34.213688 2271 ts_tablet_manager.cc:1428] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260808 14:41:34.214020 2268 ts_tablet_manager.cc:1428] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c: Time spent starting tablet: real 0.006s user 0.002s sys 0.006s
I20260808 14:41:34.214062 2114 heartbeater.cc:503] Master 127.19.144.126:43211 was elected leader, sending a full tablet report...
I20260808 14:41:34.214329 2270 ts_tablet_manager.cc:1428] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 14:41:34.214366 1978 heartbeater.cc:503] Master 127.19.144.126:43211 was elected leader, sending a full tablet report...
I20260808 14:41:34.230651 2277 raft_consensus.cc:482] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:41:34.230933 2277 raft_consensus.cc:504] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } }
I20260808 14:41:34.232364 2277 leader_election.cc:288] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605), 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67:39659)
I20260808 14:41:34.236663 2266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.233061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:38239 (local address 127.19.144.67:39659)
0808 14:41:34.233192 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:34.233199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:34.233217 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:41:34.233287 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:34.233290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:34.233338 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:34.233417 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:34.233814 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.234362 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.235600 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.236015 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.236152 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.236217 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:34.236434 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 14:41:34.236522 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:41:34.236680 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.232733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:55999 (local address 127.19.144.66:35605)
0808 14:41:34.232889 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:34.232896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:34.233077 (+ 181us) server_negotiation.cc:407] Connection header received
0808 14:41:34.233293 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:34.233296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:34.233347 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:34.233398 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:34.233910 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.234393 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.235603 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.236015 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.236152 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.236214 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:34.236493 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 14:41:34.236573 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 14:41:34.236685 2278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.232585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:35605 (local address 127.19.144.65:55999)
0808 14:41:34.232979 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:34.232996 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:34.233163 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:34.233558 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:34.233567 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:34.233582 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:34.233774 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.233783 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.234548 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.234551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.235463 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.235473 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.236152 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.236156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.236334 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.236354 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 14:41:34.236452 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 14:41:34.236522 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":167,"thread_start_us":146,"threads_started":1}
I20260808 14:41:34.236694 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.232618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:39659 (local address 127.19.144.65:38239)
0808 14:41:34.233069 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:34.233083 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:34.233184 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:34.233452 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:34.233455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:34.233474 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:34.233677 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.233686 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.234478 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.234485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.235464 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.235475 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.236152 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.236156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.236286 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.236329 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 14:41:34.236389 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 14:41:34.236489 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":34,"thread_start_us":138,"threads_started":1}
I20260808 14:41:34.237469 2205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf58191b9d71451abcf9d7981379f3b4" candidate_uuid: "e383d29721074c8db3ba1c68ac20d57c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b934e85d4ac40f2b6adf6c68526024c" is_pre_election: true
I20260808 14:41:34.237466 2066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf58191b9d71451abcf9d7981379f3b4" candidate_uuid: "e383d29721074c8db3ba1c68ac20d57c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" is_pre_election: true
I20260808 14:41:34.237833 2205 raft_consensus.cc:2454] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e383d29721074c8db3ba1c68ac20d57c in term 0.
I20260808 14:41:34.237843 2066 raft_consensus.cc:2454] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e383d29721074c8db3ba1c68ac20d57c in term 0.
I20260808 14:41:34.238366 1861 leader_election.cc:302] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7b934e85d4ac40f2b6adf6c68526024c, b8352d4aa8bc4e8fba90de4fd54ae607, e383d29721074c8db3ba1c68ac20d57c; no voters:
I20260808 14:41:34.238696 2277 raft_consensus.cc:2781] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:41:34.238804 2277 raft_consensus.cc:482] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:41:34.238850 2277 raft_consensus.cc:3037] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:34.240051 2277 raft_consensus.cc:504] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } }
I20260808 14:41:34.240588 2277 leader_election.cc:288] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 1 election: Requested vote from peers b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605), 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67:39659)
I20260808 14:41:34.241031 2066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf58191b9d71451abcf9d7981379f3b4" candidate_uuid: "e383d29721074c8db3ba1c68ac20d57c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607"
I20260808 14:41:34.241123 2205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf58191b9d71451abcf9d7981379f3b4" candidate_uuid: "e383d29721074c8db3ba1c68ac20d57c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b934e85d4ac40f2b6adf6c68526024c"
I20260808 14:41:34.241179 2066 raft_consensus.cc:3037] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:34.241331 2205 raft_consensus.cc:3037] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:34.242455 2066 raft_consensus.cc:2454] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e383d29721074c8db3ba1c68ac20d57c in term 1.
I20260808 14:41:34.242475 2205 raft_consensus.cc:2454] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e383d29721074c8db3ba1c68ac20d57c in term 1.
I20260808 14:41:34.242847 1861 leader_election.cc:302] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b934e85d4ac40f2b6adf6c68526024c, e383d29721074c8db3ba1c68ac20d57c; no voters:
I20260808 14:41:34.243057 2277 raft_consensus.cc:2781] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:41:34.243345 2277 raft_consensus.cc:686] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [term 1 LEADER]: Becoming Leader. State: Replica: e383d29721074c8db3ba1c68ac20d57c, State: Running, Role: LEADER
I20260808 14:41:34.243777 2277 consensus_queue.cc:237] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } }
I20260808 14:41:34.247424 1783 catalog_manager.cc:5495] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c reported cstate change: term changed from 0 to 1, leader changed from <none> to e383d29721074c8db3ba1c68ac20d57c (127.19.144.65). New cstate: current_term: 1 leader_uuid: "e383d29721074c8db3ba1c68ac20d57c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } health_report { overall_health: UNKNOWN } } }
I20260808 14:41:34.271368 2267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.267651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40924 (local address 127.19.144.65:36149)
0808 14:41:34.267785 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:34.267789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:34.267804 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:41:34.267884 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:34.267915 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:34.267972 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:34.268070 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:41:34.268318 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.268827 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.269770 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.270006 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.270091 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.270457 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:41:34.270974 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:41:34.271080 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:34.271190 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 14:41:34.271244 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:41:34.278071 2066 raft_consensus.cc:1261] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 1 FOLLOWER]: Refusing update from remote peer e383d29721074c8db3ba1c68ac20d57c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:41:34.278071 2205 raft_consensus.cc:1261] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c [term 1 FOLLOWER]: Refusing update from remote peer e383d29721074c8db3ba1c68ac20d57c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:41:34.278920 2277 consensus_queue.cc:1035] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:41:34.279139 2280 consensus_queue.cc:1035] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:41:34.287902 2285 mvcc.cc:205] Tried to move back new op lower bound from 7316275586154110976 to 7316275586025533440. Current Snapshot: MvccSnapshot[applied={T|T < 7316275586154110976}]
I20260808 14:41:34.287940 2284 mvcc.cc:205] Tried to move back new op lower bound from 7316275586154110976 to 7316275586025533440. Current Snapshot: MvccSnapshot[applied={T|T < 7316275586154110976}]
I20260808 14:41:34.297699 1783 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:41:34.297915 1783 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:41:34.300452 1783 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: Sending DeleteTablet for 3 replicas of tablet bf58191b9d71451abcf9d7981379f3b4
I20260808 14:41:34.301391 2183 tablet_service.cc:1514] Processing DeleteTablet for tablet bf58191b9d71451abcf9d7981379f3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:41:34 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:54372
I20260808 14:41:34.301393 1910 tablet_service.cc:1514] Processing DeleteTablet for tablet bf58191b9d71451abcf9d7981379f3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:41:34 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:40908
I20260808 14:41:34.301537 2046 tablet_service.cc:1514] Processing DeleteTablet for tablet bf58191b9d71451abcf9d7981379f3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:41:34 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:54510
I20260808 14:41:34.302119 2296 tablet_replica.cc:330] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607: stopping tablet replica
I20260808 14:41:34.302139 2295 tablet_replica.cc:330] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c: stopping tablet replica
I20260808 14:41:34.302232 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 1845
I20260808 14:41:34.302479 2295 raft_consensus.cc:2229] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:41:34.302488 2296 raft_consensus.cc:2229] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:41:34.302825 2295 raft_consensus.cc:2258] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:41:34.302827 2296 raft_consensus.cc:2258] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:41:34.304592 2296 ts_tablet_manager.cc:1905] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:41:34.304592 2295 ts_tablet_manager.cc:1905] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:41:34.307309 2296 ts_tablet_manager.cc:1918] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:41:34.307413 2296 log.cc:1196] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/bf58191b9d71451abcf9d7981379f3b4
I20260808 14:41:34.307758 2296 ts_tablet_manager.cc:1939] T bf58191b9d71451abcf9d7981379f3b4 P b8352d4aa8bc4e8fba90de4fd54ae607: Deleting consensus metadata
I20260808 14:41:34.308293 2295 ts_tablet_manager.cc:1918] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:41:34.308401 2295 log.cc:1196] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/bf58191b9d71451abcf9d7981379f3b4
I20260808 14:41:34.308776 2295 ts_tablet_manager.cc:1939] T bf58191b9d71451abcf9d7981379f3b4 P 7b934e85d4ac40f2b6adf6c68526024c: Deleting consensus metadata
I20260808 14:41:34.309130 1770 catalog_manager.cc:4841] TS b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605): tablet bf58191b9d71451abcf9d7981379f3b4 (table test-table [id=eede6667c3db45359499406907cac6b1]) successfully deleted
I20260808 14:41:34.310177 1767 catalog_manager.cc:4841] TS 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67:39659): tablet bf58191b9d71451abcf9d7981379f3b4 (table test-table [id=eede6667c3db45359499406907cac6b1]) successfully deleted
W20260808 14:41:34.312901 1770 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.144.65:36149 (error 108)
I20260808 14:41:34.312966 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 1983
I20260808 14:41:34.314963 2265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.314654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:36149 (local address 127.0.0.1:40936)
0808 14:41:34.314794 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:34.314880 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:36149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 14:41:34.315171 1770 catalog_manager.cc:4584] TS e383d29721074c8db3ba1c68ac20d57c (127.19.144.65:36149): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf58191b9d71451abcf9d7981379f3b4: Network error: Client connection negotiation failed: client connection to 127.19.144.65:36149: connect: Connection refused (error 111)
I20260808 14:41:34.320605 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 2119
I20260808 14:41:34.330375 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 1751
I20260808 14:41:34.339344 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:43211
--webserver_interface=127.19.144.126
--webserver_port=34367
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:43211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:41:34.458446 2298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:34.458762 2298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:34.458804 2298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:34.463099 2298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:41:34.463173 2298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:34.463193 2298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:34.463210 2298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:41:34.463227 2298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:41:34.469503 2298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:43211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:43211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=34367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:34.470775 2298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:34.471906 2298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:34.486632 2305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:34.486655 2304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:34.486763 2298 server_base.cc:1034] running on GCE node
W20260808 14:41:34.486634 2307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:34.487682 2298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:34.488904 2298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:34.490103 2298 hybrid_clock.cc:648] HybridClock initialized: now 1786200094490077 us; error 39 us; skew 500 ppm
Aug 08 14:41:34 dist-test-slave-p3t5 krb5kdc[1737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200094, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:34.493016 2298 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:41:34.494107 2298 webserver.cc:460] Webserver started at http://127.19.144.126:34367/ using document root <none> and password file <none>
I20260808 14:41:34.494719 2298 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:34.494894 2298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:34.498996 2298 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:41:34.501260 2314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:34.502704 2298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:41:34.502882 2298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "bf92c5ea5ec240578d9e6f8f4b227c26"
format_stamp: "Formatted at 2026-08-08 14:41:33 on dist-test-slave-p3t5"
server_key: "a4a80bb651ede6c22dc4f6e53d29a87f"
server_key_iv: "1cff0132a8395ea3a72b3e7b2d4828eb"
server_key_version: "encryptionkey@0"
I20260808 14:41:34.503559 2298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:34.537784 2298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:34.541435 2298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:34.541762 2298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:34.549794 2298 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:43211
I20260808 14:41:34.550113 2368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:43211 every 8 connection(s)
I20260808 14:41:34.551124 2298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:41:34.556151 2369 sys_catalog.cc:263] Verifying existing consensus state
I20260808 14:41:34.557964 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 2298
I20260808 14:41:34.558578 2369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: Bootstrap starting.
I20260808 14:41:34.558594 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:36149
--local_ip_for_outbound_sockets=127.19.144.65
--tserver_master_addrs=127.19.144.126:43211
--webserver_port=36391
--webserver_interface=127.19.144.65
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:41:34.579826 2369 log.cc:824] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:34.589551 2369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:41:34.590082 2369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: Bootstrap complete.
I20260808 14:41:34.594547 2369 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } }
I20260808 14:41:34.595041 2369 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf92c5ea5ec240578d9e6f8f4b227c26, State: Initialized, Role: FOLLOWER
I20260808 14:41:34.595652 2369 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } }
I20260808 14:41:34.595826 2369 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:41:34.595920 2369 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:41:34.596052 2369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 1 FOLLOWER]: Advancing to term 2
I20260808 14:41:34.598302 2369 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } }
I20260808 14:41:34.598692 2369 leader_election.cc:302] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf92c5ea5ec240578d9e6f8f4b227c26; no voters:
I20260808 14:41:34.599093 2369 leader_election.cc:288] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 14:41:34.599212 2373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 14:41:34.599443 2373 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 2 LEADER]: Becoming Leader. State: Replica: bf92c5ea5ec240578d9e6f8f4b227c26, State: Running, Role: LEADER
I20260808 14:41:34.599764 2373 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } }
I20260808 14:41:34.600474 2369 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:41:34.600859 2375 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf92c5ea5ec240578d9e6f8f4b227c26. Latest consensus state: current_term: 2 leader_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } } }
I20260808 14:41:34.601001 2375 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [sys.catalog]: This master's current role is: LEADER
I20260808 14:41:34.601228 2374 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } } }
I20260808 14:41:34.601333 2374 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [sys.catalog]: This master's current role is: LEADER
I20260808 14:41:34.601418 2377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:41:34.606801 2377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:41:34.607796 2377 catalog_manager.cc:1259] Loaded cluster ID: b9c9317e9cd846149f0952d7300db951
I20260808 14:41:34.607867 2377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:41:34.609376 2377 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 14:41:34.614336 2377 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 14:41:34.620332 2377 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:41:34.621222 2377 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: Loaded TSK: 0
I20260808 14:41:34.622562 2377 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 14:41:34.686211 2371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:34.686529 2371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:34.686616 2371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:34.690835 2371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:34.690937 2371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:34.691035 2371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:41:34.696473 2371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:36149
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=36391
--tserver_master_addrs=127.19.144.126:43211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:34.697759 2371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:34.698825 2371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:34.710529 2398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:34.710529 2397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:34.710529 2400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:34.711784 2371 server_base.cc:1034] running on GCE node
I20260808 14:41:34.712236 2371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:34.712925 2371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:34.714128 2371 hybrid_clock.cc:648] HybridClock initialized: now 1786200094714090 us; error 54 us; skew 500 ppm
Aug 08 14:41:34 dist-test-slave-p3t5 krb5kdc[1737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200094, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:34.717252 2371 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:41:34.718372 2371 webserver.cc:460] Webserver started at http://127.19.144.65:36391/ using document root <none> and password file <none>
I20260808 14:41:34.718986 2371 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:34.719108 2371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:34.723595 2371 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 14:41:34.726257 2407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:34.727831 2371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:41:34.728000 2371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "e383d29721074c8db3ba1c68ac20d57c"
format_stamp: "Formatted at 2026-08-08 14:41:33 on dist-test-slave-p3t5"
server_key: "2e1e830b9634ec987cd2a84d56900712"
server_key_iv: "33c17947ca736e24b03300a8d816ce58"
server_key_version: "encryptionkey@0"
I20260808 14:41:34.728626 2371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:34.742405 2371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:34.746148 2371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:34.746381 2371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:34.747059 2371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:34.748404 2414 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 14:41:34.752872 2371 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 14:41:34.752962 2371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260808 14:41:34.753049 2371 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 14:41:34.755028 2371 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 14:41:34.755101 2371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260808 14:41:34.755376 2414 tablet_bootstrap.cc:492] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c: Bootstrap starting.
I20260808 14:41:34.766999 2371 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:36149
I20260808 14:41:34.767059 2523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:36149 every 8 connection(s)
I20260808 14:41:34.768428 2371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:41:34.768805 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 2371
I20260808 14:41:34.769333 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:35605
--local_ip_for_outbound_sockets=127.19.144.66
--tserver_master_addrs=127.19.144.126:43211
--webserver_port=45029
--webserver_interface=127.19.144.66
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:41:34.778187 2414 log.cc:824] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:41:34 dist-test-slave-p3t5 krb5kdc[1737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200094, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:34.789247 2414 tablet_bootstrap.cc:492] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:41:34.789680 2527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.771514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:34817 (local address 127.19.144.126:43211)
0808 14:41:34.772131 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:34.772165 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:34.773924 (+ 1759us) server_negotiation.cc:407] Connection header received
0808 14:41:34.775186 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:34.775208 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:34.775734 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:34.776114 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:34.777984 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.779322 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.780557 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.781067 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.781284 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.784074 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:34.784080 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:34.784094 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:34.784126 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:34.786056 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:34.786682 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:34.786687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:34.786692 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:34.786846 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:34.787232 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:34.787235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:34.787236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:34.787678 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:34.787833 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:34.788396 (+ 563us) server_negotiation.cc:299] Negotiation successful
0808 14:41:34.788663 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":129,"threads_started":1}
I20260808 14:41:34.789690 2528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.771515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:34817)
0808 14:41:34.773200 (+ 1685us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:34.773264 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:34.774627 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:34.776613 (+ 1986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:34.776623 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:34.777108 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:34.777770 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.777791 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.779517 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.779520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.780409 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.780420 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.781243 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.781247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.781448 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.782198 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:34.782216 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:34.783850 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:34.786225 (+ 2375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:34.786231 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:34.786245 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:34.786538 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:34.786952 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:34.786955 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:34.786957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:34.787116 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:34.787798 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:34.787805 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:34.788068 (+ 263us) client_negotiation.cc:769] Sending connection context
0808 14:41:34.788255 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 14:41:34.788636 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1437,"thread_start_us":164,"threads_started":1}
I20260808 14:41:34.789765 2414 tablet_bootstrap.cc:492] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c: Bootstrap complete.
I20260808 14:41:34.791256 2414 ts_tablet_manager.cc:1397] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c: Time spent bootstrapping tablet: real 0.036s user 0.018s sys 0.015s
I20260808 14:41:34.792423 2524 heartbeater.cc:347] Connected to a master server at 127.19.144.126:43211
I20260808 14:41:34.792694 2524 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:34.793294 2524 heartbeater.cc:511] Master 127.19.144.126:43211 requested a full tablet report, sending...
I20260808 14:41:34.794346 2414 raft_consensus.cc:348] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } }
I20260808 14:41:34.794994 2414 raft_consensus.cc:729] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e383d29721074c8db3ba1c68ac20d57c, State: Initialized, Role: FOLLOWER
I20260808 14:41:34.795502 2414 consensus_queue.cc:260] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } }
I20260808 14:41:34.796226 2331 ts_manager.cc:194] Registered new tserver with Master: e383d29721074c8db3ba1c68ac20d57c (127.19.144.65:36149)
I20260808 14:41:34.796568 2414 ts_tablet_manager.cc:1428] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260808 14:41:34.800493 2331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:34817
I20260808 14:41:34.803442 2524 heartbeater.cc:503] Master 127.19.144.126:43211 was elected leader, sending a full tablet report...
Aug 08 14:41:34 dist-test-slave-p3t5 krb5kdc[1737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200094, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for kudu/127.19.144.65@KRBTEST.COM
I20260808 14:41:34.812885 2532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.799596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40950 (local address 127.19.144.65:36149)
0808 14:41:34.801476 (+ 1880us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:34.801494 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:34.801518 (+ 24us) server_negotiation.cc:407] Connection header received
0808 14:41:34.801587 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:34.801593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:34.801847 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:34.801995 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:34.802868 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.803865 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.805084 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.805547 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.805732 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.808676 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:34.808687 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:34.808699 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:34.808727 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:34.810699 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:34.811295 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:34.811301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:34.811302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:34.811398 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:34.811728 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:34.811731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:34.811733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:34.812037 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:34.812150 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:34.812547 (+ 397us) server_negotiation.cc:299] Negotiation successful
0808 14:41:34.812704 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1763,"thread_start_us":82,"threads_started":1}
I20260808 14:41:34.812896 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.799484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:36149 (local address 127.0.0.1:40950)
0808 14:41:34.799964 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:34.800018 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:34.800174 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:34.802072 (+ 1898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:34.802078 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:34.802551 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:34.802693 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.802702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.804004 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.804009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.804963 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.804975 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.805756 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.805760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.805878 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.806550 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:34.806566 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:34.808495 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:34.810827 (+ 2332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:34.810838 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:34.810844 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:34.811165 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:34.811497 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:34.811500 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:34.811502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:34.811608 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:34.812162 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:34.812169 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:34.812309 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 14:41:34.812462 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 14:41:34.812701 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":114,"threads_started":1}
I20260808 14:41:34.814492 2456 tablet_service.cc:1514] Processing DeleteTablet for tablet bf58191b9d71451abcf9d7981379f3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:41:34 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:40950
I20260808 14:41:34.815631 2533 tablet_replica.cc:330] stopping tablet replica
I20260808 14:41:34.815970 2533 raft_consensus.cc:2229] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:41:34.816244 2533 raft_consensus.cc:2258] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:41:34.817252 2533 ts_tablet_manager.cc:1905] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:41:34.821076 2533 ts_tablet_manager.cc:1918] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:41:34.821205 2533 log.cc:1196] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal/wals/bf58191b9d71451abcf9d7981379f3b4
I20260808 14:41:34.821502 2533 ts_tablet_manager.cc:1939] T bf58191b9d71451abcf9d7981379f3b4 P e383d29721074c8db3ba1c68ac20d57c: Deleting consensus metadata
I20260808 14:41:34.823385 2318 catalog_manager.cc:4841] TS e383d29721074c8db3ba1c68ac20d57c (127.19.144.65:36149): tablet bf58191b9d71451abcf9d7981379f3b4 (table test-table [id=eede6667c3db45359499406907cac6b1]) successfully deleted
W20260808 14:41:34.904713 2526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:34.905004 2526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:34.905110 2526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:34.909449 2526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:34.909515 2526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:34.909576 2526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:41:34.914989 2526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:35605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=45029
--tserver_master_addrs=127.19.144.126:43211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:34.916096 2526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:34.917124 2526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:34.927770 2539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:34.927798 2541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:34.927716 2538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:34.927743 2526 server_base.cc:1034] running on GCE node
I20260808 14:41:34.928369 2526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:34.929064 2526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:34.930251 2526 hybrid_clock.cc:648] HybridClock initialized: now 1786200094930233 us; error 32 us; skew 500 ppm
Aug 08 14:41:34 dist-test-slave-p3t5 krb5kdc[1737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200094, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:34.933300 2526 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:41:34.934329 2526 webserver.cc:460] Webserver started at http://127.19.144.66:45029/ using document root <none> and password file <none>
I20260808 14:41:34.934944 2526 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:34.935024 2526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:34.939482 2526 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:41:34.941762 2548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:34.943104 2526 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:41:34.943249 2526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "b8352d4aa8bc4e8fba90de4fd54ae607"
format_stamp: "Formatted at 2026-08-08 14:41:33 on dist-test-slave-p3t5"
server_key: "b9da2594f4341d8a39eaf6f317e9588c"
server_key_iv: "53ffacb03f8f93be1e604fd432559d79"
server_key_version: "encryptionkey@0"
I20260808 14:41:34.943823 2526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:34.958803 2526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:34.962529 2526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:34.962785 2526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:34.963454 2526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:34.964385 2526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:41:34.964433 2526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:34.964499 2526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:41:34.964536 2526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:34.974632 2665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:35605 every 8 connection(s)
I20260808 14:41:34.974647 2526 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:35605
I20260808 14:41:34.975929 2526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:41:34.981602 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 2526
I20260808 14:41:34.982128 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:39659
--local_ip_for_outbound_sockets=127.19.144.67
--tserver_master_addrs=127.19.144.126:43211
--webserver_port=38335
--webserver_interface=127.19.144.67
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:41:34 dist-test-slave-p3t5 krb5kdc[1737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200094, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:34.991811 2527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.978065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:45533 (local address 127.19.144.126:43211)
0808 14:41:34.978230 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:34.978234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:34.979154 (+ 920us) server_negotiation.cc:407] Connection header received
0808 14:41:34.980658 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:34.980662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:34.980733 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:34.980854 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:34.982586 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.983121 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.984721 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.984974 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.985097 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.987885 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:34.987890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:34.987893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:34.987929 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:34.989500 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:34.990095 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:34.990098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:34.990100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:34.990196 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:34.990577 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:34.990581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:34.990582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:34.990795 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:34.990911 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:34.991458 (+ 547us) server_negotiation.cc:299] Negotiation successful
0808 14:41:34.991609 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 14:41:34.992681 2668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:34.978510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:45533)
0808 14:41:34.978996 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:34.979043 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:34.980423 (+ 1380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:34.981063 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:34.981071 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:34.981601 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:34.982395 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.982422 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.983278 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.983285 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.984563 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:34.984574 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:34.985146 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:34.985151 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:34.985349 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:34.986038 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:34.986055 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:34.987685 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:34.989660 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:34.989667 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:34.989684 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:34.989976 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:34.990303 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:34.990306 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:34.990308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:34.990465 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:34.990911 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:34.990919 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:34.991189 (+ 270us) client_negotiation.cc:769] Sending connection context
0808 14:41:34.991396 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 14:41:34.991757 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":108,"threads_started":1}
I20260808 14:41:34.994022 2666 heartbeater.cc:347] Connected to a master server at 127.19.144.126:43211
I20260808 14:41:34.994311 2666 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:34.994874 2666 heartbeater.cc:511] Master 127.19.144.126:43211 requested a full tablet report, sending...
I20260808 14:41:34.996054 2331 ts_manager.cc:194] Registered new tserver with Master: b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605)
I20260808 14:41:34.996837 2331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:45533
W20260808 14:41:35.104637 2669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:35.104938 2669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:35.104983 2669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:35.109560 2669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:35.109719 2669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:35.109792 2669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:41:35.115350 2669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:39659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=38335
--tserver_master_addrs=127.19.144.126:43211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:35.116580 2669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:35.117837 2669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:35.128566 2677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:35.128575 2674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:35.128738 2669 server_base.cc:1034] running on GCE node
W20260808 14:41:35.128573 2675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:35.129408 2669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:35.130162 2669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:35.131358 2669 hybrid_clock.cc:648] HybridClock initialized: now 1786200095131339 us; error 31 us; skew 500 ppm
Aug 08 14:41:35 dist-test-slave-p3t5 krb5kdc[1737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200095, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:35.134677 2669 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:41:35.135600 2669 webserver.cc:460] Webserver started at http://127.19.144.67:38335/ using document root <none> and password file <none>
I20260808 14:41:35.136158 2669 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:35.136235 2669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:35.140451 2669 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260808 14:41:35.142693 2685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:35.144152 2669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:41:35.144321 2669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "7b934e85d4ac40f2b6adf6c68526024c"
format_stamp: "Formatted at 2026-08-08 14:41:34 on dist-test-slave-p3t5"
server_key: "5ceedeae0ac68b7836644c83bec9cc59"
server_key_iv: "3e0592cf1e3b2427c3fd5c449aac45a9"
server_key_version: "encryptionkey@0"
I20260808 14:41:35.144927 2669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:35.160545 2669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:35.163885 2669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:35.164211 2669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:35.164882 2669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:35.165899 2669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:41:35.165951 2669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:35.166023 2669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:41:35.166062 2669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:35.175338 2669 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:39659
I20260808 14:41:35.175352 2799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:39659 every 8 connection(s)
I20260808 14:41:35.176455 2669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:41:35.180490 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 2669
Aug 08 14:41:35 dist-test-slave-p3t5 krb5kdc[1737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200095, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:35.191262 2805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:35.181249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43394 (local address 127.19.144.126:43211)
0808 14:41:35.181479 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:35.181483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:35.181765 (+ 282us) server_negotiation.cc:407] Connection header received
0808 14:41:35.181885 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:35.181887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:35.181923 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:35.181996 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:35.182918 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.183890 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.185135 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.185375 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.185491 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:35.187139 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:35.187143 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:35.187145 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:35.187171 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:35.189163 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:35.189799 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:35.189807 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:35.189809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:35.189913 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:35.190266 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:35.190269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:35.190270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:35.190466 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:35.190714 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:35.190916 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 14:41:35.191051 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":158,"thread_start_us":71,"threads_started":1}
I20260808 14:41:35.192538 2527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:35.178381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:55939 (local address 127.19.144.126:43211)
0808 14:41:35.178534 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:35.178538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:35.179400 (+ 862us) server_negotiation.cc:407] Connection header received
0808 14:41:35.181270 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:35.181274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:35.181344 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:35.181430 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:35.183279 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.183975 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.185510 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.185844 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.185960 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:35.188696 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:35.188700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:35.188703 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:35.188745 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:35.190400 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:35.190919 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:35.190922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:35.190924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:35.191008 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:35.191355 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:35.191360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:35.191362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:35.191555 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:35.191661 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:35.192260 (+ 599us) server_negotiation.cc:299] Negotiation successful
0808 14:41:35.192378 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":55}
I20260808 14:41:35.193594 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:35.178721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:55939)
0808 14:41:35.179222 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:35.179272 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:35.181006 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:35.181704 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:35.181717 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:35.182310 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:35.183035 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:35.183055 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.184098 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.184103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:35.185370 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:35.185380 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.186014 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.186020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:35.186199 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:35.186772 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:35.186792 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:35.188526 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:35.190512 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:35.190519 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:35.190536 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:35.190813 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:35.191101 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:35.191105 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:35.191107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:35.191259 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:35.191676 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:35.191681 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:35.191950 (+ 269us) client_negotiation.cc:769] Sending connection context
0808 14:41:35.192196 (+ 246us) client_negotiation.cc:240] Negotiation successful
0808 14:41:35.192567 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":116,"threads_started":1}
I20260808 14:41:35.194830 2800 heartbeater.cc:347] Connected to a master server at 127.19.144.126:43211
I20260808 14:41:35.195103 2800 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:35.195633 2800 heartbeater.cc:511] Master 127.19.144.126:43211 requested a full tablet report, sending...
I20260808 14:41:35.196653 2331 ts_manager.cc:194] Registered new tserver with Master: 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67:39659)
I20260808 14:41:35.197217 2331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:55939
I20260808 14:41:35.205466 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:41:35.210739 2527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:35.206197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43408 (local address 127.19.144.126:43211)
0808 14:41:35.206400 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:35.206407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:35.206596 (+ 189us) server_negotiation.cc:407] Connection header received
0808 14:41:35.206919 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:35.206922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:35.207041 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:35.207166 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:41:35.207465 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.208050 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.209058 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.209312 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.209437 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:35.209832 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:41:35.210283 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:41:35.210438 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:35.210508 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 14:41:35.210571 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260808 14:41:35.211476 2331 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:43408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:41:35.215158 2331 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:41:35.223745 2456 tablet_service.cc:1467] Processing CreateTablet for tablet e585b1c4774b4d398c9e801ad6f7ac92 (DEFAULT_TABLE table=test-table [id=6f62c775a3654d16ae5b51366db4aa3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:41:35.224495 2456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e585b1c4774b4d398c9e801ad6f7ac92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:35.227488 2414 tablet_bootstrap.cc:492] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c: Bootstrap starting.
I20260808 14:41:35.227963 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:35.222311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:35605 (local address 127.0.0.1:54516)
0808 14:41:35.222448 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:35.222466 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:35.222548 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:35.223626 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:35.223629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:35.223647 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:35.223770 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:35.223779 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.225163 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.225166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:35.226322 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:35.226333 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.227497 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.227501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:35.227611 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:35.227667 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 14:41:35.227721 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 14:41:35.227809 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 14:41:35.228434 2414 tablet_bootstrap.cc:654] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:35.228767 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:35.222491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54516 (local address 127.19.144.66:35605)
0808 14:41:35.223100 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:35.223118 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:35.223146 (+ 28us) server_negotiation.cc:407] Connection header received
0808 14:41:35.223245 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:35.223251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:35.223475 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:35.223647 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:35.223899 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.225001 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.226478 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.227358 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.228308 (+ 950us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:35.228468 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:35.228521 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 14:41:35.228614 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":154,"threads_started":1}
I20260808 14:41:35.229687 2811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:35.222692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:39659 (local address 127.0.0.1:54378)
0808 14:41:35.223167 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:35.223181 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:35.223267 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:35.223930 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:35.224079 (+ 149us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:35.224101 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:35.224196 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:35.224203 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.225965 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.225968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:35.227118 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:35.227129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.229229 (+ 2100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.229233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:35.229335 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:35.229359 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:41:35.229428 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 14:41:35.229509 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":97,"threads_started":1}
I20260808 14:41:35.230078 2812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:35.222875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54378 (local address 127.19.144.67:39659)
0808 14:41:35.223432 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:35.223445 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:35.223469 (+ 24us) server_negotiation.cc:407] Connection header received
0808 14:41:35.223548 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:35.223553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:35.223743 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:35.223953 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:35.224331 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.225807 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.227268 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.228917 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.229031 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:35.229165 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:35.229867 (+ 702us) server_negotiation.cc:299] Negotiation successful
0808 14:41:35.229970 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":176,"threads_started":1}
I20260808 14:41:35.231621 2414 tablet_bootstrap.cc:492] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c: No bootstrap required, opened a new log
I20260808 14:41:35.231727 2414 ts_tablet_manager.cc:1397] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260808 14:41:35.232216 2414 raft_consensus.cc:348] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } }
I20260808 14:41:35.232354 2414 raft_consensus.cc:374] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:35.232451 2414 raft_consensus.cc:729] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e383d29721074c8db3ba1c68ac20d57c, State: Initialized, Role: FOLLOWER
I20260808 14:41:35.232115 2598 tablet_service.cc:1467] Processing CreateTablet for tablet e585b1c4774b4d398c9e801ad6f7ac92 (DEFAULT_TABLE table=test-table [id=6f62c775a3654d16ae5b51366db4aa3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:41:35.232604 2414 consensus_queue.cc:260] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } }
I20260808 14:41:35.232874 2414 ts_tablet_manager.cc:1428] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 14:41:35.233108 2733 tablet_service.cc:1467] Processing CreateTablet for tablet e585b1c4774b4d398c9e801ad6f7ac92 (DEFAULT_TABLE table=test-table [id=6f62c775a3654d16ae5b51366db4aa3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:41:35.234052 2598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e585b1c4774b4d398c9e801ad6f7ac92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:35.234151 2733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e585b1c4774b4d398c9e801ad6f7ac92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:35.241302 2815 tablet_bootstrap.cc:492] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607: Bootstrap starting.
I20260808 14:41:35.241285 2814 tablet_bootstrap.cc:492] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c: Bootstrap starting.
I20260808 14:41:35.243806 2814 tablet_bootstrap.cc:654] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:35.244190 2815 tablet_bootstrap.cc:654] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:35.244946 2814 log.cc:824] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:35.244952 2815 log.cc:824] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:35.247092 2815 tablet_bootstrap.cc:492] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607: No bootstrap required, opened a new log
I20260808 14:41:35.247098 2814 tablet_bootstrap.cc:492] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c: No bootstrap required, opened a new log
I20260808 14:41:35.247326 2814 ts_tablet_manager.cc:1397] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:41:35.247320 2815 ts_tablet_manager.cc:1397] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:41:35.250860 2814 raft_consensus.cc:348] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } }
I20260808 14:41:35.250883 2815 raft_consensus.cc:348] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } }
I20260808 14:41:35.251120 2814 raft_consensus.cc:374] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:35.251113 2815 raft_consensus.cc:374] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:35.251181 2815 raft_consensus.cc:729] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8352d4aa8bc4e8fba90de4fd54ae607, State: Initialized, Role: FOLLOWER
I20260808 14:41:35.251183 2814 raft_consensus.cc:729] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b934e85d4ac40f2b6adf6c68526024c, State: Initialized, Role: FOLLOWER
I20260808 14:41:35.251646 2814 consensus_queue.cc:260] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } }
I20260808 14:41:35.251646 2815 consensus_queue.cc:260] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } }
I20260808 14:41:35.252345 2800 heartbeater.cc:503] Master 127.19.144.126:43211 was elected leader, sending a full tablet report...
I20260808 14:41:35.252684 2814 ts_tablet_manager.cc:1428] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 14:41:35.252701 2815 ts_tablet_manager.cc:1428] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 14:41:35.253008 2666 heartbeater.cc:503] Master 127.19.144.126:43211 was elected leader, sending a full tablet report...
W20260808 14:41:35.270216 2525 tablet.cc:2369] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:41:35.427533 2802 tablet.cc:2369] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:41:35.432312 2818 raft_consensus.cc:482] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:41:35.432575 2818 raft_consensus.cc:504] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } }
I20260808 14:41:35.434073 2818 leader_election.cc:288] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605), e383d29721074c8db3ba1c68ac20d57c (127.19.144.65:36149)
I20260808 14:41:35.439181 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:35.434660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:59051 (local address 127.19.144.66:35605)
0808 14:41:35.434886 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:35.434892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:35.434909 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:41:35.434967 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:35.434970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:35.435028 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:35.435113 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:35.435512 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.436266 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.437981 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.438554 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.438682 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:35.438737 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:35.438965 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 14:41:35.439045 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260808 14:41:35.439188 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:35.434385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:35605 (local address 127.19.144.67:59051)
0808 14:41:35.434544 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:35.434561 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:35.434666 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:35.435163 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:35.435169 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:35.435201 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:35.435368 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:35.435376 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.436401 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.436404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:35.437845 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:35.437857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.438687 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.438690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:35.438794 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:35.438818 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:41:35.438931 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 14:41:35.439017 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":31}
I20260808 14:41:35.439416 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:35.434690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:36149 (local address 127.19.144.67:35999)
0808 14:41:35.435163 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:35.435178 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:35.435296 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:35.435676 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:35.435678 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:35.435697 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:35.435833 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:35.435838 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.436743 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.436747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:35.438061 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:35.438069 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.438949 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.438952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:35.439067 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:35.439092 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 14:41:35.439174 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 14:41:35.439251 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":1,"thread_start_us":163,"threads_started":1}
I20260808 14:41:35.439419 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:35.434834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:35999 (local address 127.19.144.65:36149)
0808 14:41:35.435143 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:35.435148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:35.435277 (+ 129us) server_negotiation.cc:407] Connection header received
0808 14:41:35.435397 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:35.435400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:35.435496 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:35.435621 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:35.435979 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.436601 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.438170 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.438819 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.438960 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:35.439118 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:35.439192 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 14:41:35.439273 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":123,"threads_started":1}
I20260808 14:41:35.440047 2619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e585b1c4774b4d398c9e801ad6f7ac92" candidate_uuid: "7b934e85d4ac40f2b6adf6c68526024c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" is_pre_election: true
I20260808 14:41:35.440023 2476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e585b1c4774b4d398c9e801ad6f7ac92" candidate_uuid: "7b934e85d4ac40f2b6adf6c68526024c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e383d29721074c8db3ba1c68ac20d57c" is_pre_election: true
I20260808 14:41:35.440376 2619 raft_consensus.cc:2454] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b934e85d4ac40f2b6adf6c68526024c in term 0.
I20260808 14:41:35.440383 2476 raft_consensus.cc:2454] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b934e85d4ac40f2b6adf6c68526024c in term 0.
I20260808 14:41:35.440904 2689 leader_election.cc:302] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b934e85d4ac40f2b6adf6c68526024c, e383d29721074c8db3ba1c68ac20d57c; no voters:
I20260808 14:41:35.441275 2818 raft_consensus.cc:2781] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:41:35.441378 2818 raft_consensus.cc:482] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:41:35.441429 2818 raft_consensus.cc:3037] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:35.442677 2818 raft_consensus.cc:504] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } }
I20260808 14:41:35.443077 2818 leader_election.cc:288] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [CANDIDATE]: Term 1 election: Requested vote from peers b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605), e383d29721074c8db3ba1c68ac20d57c (127.19.144.65:36149)
I20260808 14:41:35.443519 2619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e585b1c4774b4d398c9e801ad6f7ac92" candidate_uuid: "7b934e85d4ac40f2b6adf6c68526024c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607"
I20260808 14:41:35.443580 2476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e585b1c4774b4d398c9e801ad6f7ac92" candidate_uuid: "7b934e85d4ac40f2b6adf6c68526024c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e383d29721074c8db3ba1c68ac20d57c"
I20260808 14:41:35.443689 2619 raft_consensus.cc:3037] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:35.443722 2476 raft_consensus.cc:3037] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:35.444772 2476 raft_consensus.cc:2454] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b934e85d4ac40f2b6adf6c68526024c in term 1.
I20260808 14:41:35.444854 2619 raft_consensus.cc:2454] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b934e85d4ac40f2b6adf6c68526024c in term 1.
I20260808 14:41:35.445114 2689 leader_election.cc:302] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b934e85d4ac40f2b6adf6c68526024c, e383d29721074c8db3ba1c68ac20d57c; no voters:
I20260808 14:41:35.445317 2818 raft_consensus.cc:2781] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:41:35.445564 2818 raft_consensus.cc:686] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [term 1 LEADER]: Becoming Leader. State: Replica: 7b934e85d4ac40f2b6adf6c68526024c, State: Running, Role: LEADER
I20260808 14:41:35.445981 2818 consensus_queue.cc:237] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } }
I20260808 14:41:35.450054 2331 catalog_manager.cc:5495] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67). New cstate: current_term: 1 leader_uuid: "7b934e85d4ac40f2b6adf6c68526024c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } health_report { overall_health: UNKNOWN } } }
I20260808 14:41:35.470292 2812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:35.466662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53572 (local address 127.19.144.67:39659)
0808 14:41:35.466811 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:35.466815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:35.466829 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:41:35.466954 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:35.466956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:35.467015 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:35.467108 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:41:35.467345 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.467856 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.468856 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:35.469076 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:35.469158 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:35.469493 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:41:35.469947 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:41:35.470037 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:35.470101 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 14:41:35.470154 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 14:41:35.476662 2476 raft_consensus.cc:1261] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Refusing update from remote peer 7b934e85d4ac40f2b6adf6c68526024c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:41:35.476665 2619 raft_consensus.cc:1261] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 1 FOLLOWER]: Refusing update from remote peer 7b934e85d4ac40f2b6adf6c68526024c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260808 14:41:35.477116 2667 tablet.cc:2369] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:41:35.477479 2822 consensus_queue.cc:1035] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:41:35.477743 2818 consensus_queue.cc:1035] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:41:35.486146 2826 mvcc.cc:205] Tried to move back new op lower bound from 7316275591064010752 to 7316275590949433344. Current Snapshot: MvccSnapshot[applied={T|T < 7316275591064010752}]
I20260808 14:41:35.487584 2827 mvcc.cc:205] Tried to move back new op lower bound from 7316275591064010752 to 7316275590949433344. Current Snapshot: MvccSnapshot[applied={T|T < 7316275591064010752}]
I20260808 14:41:35.492200 2331 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:43408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:41:35.492398 2331 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:43408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:41:35.494895 2331 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: Sending DeleteTablet for 3 replicas of tablet e585b1c4774b4d398c9e801ad6f7ac92
I20260808 14:41:35.495579 2598 tablet_service.cc:1514] Processing DeleteTablet for tablet e585b1c4774b4d398c9e801ad6f7ac92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:41:35 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:54516
I20260808 14:41:35.495730 2456 tablet_service.cc:1514] Processing DeleteTablet for tablet e585b1c4774b4d398c9e801ad6f7ac92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:41:35 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:40950
I20260808 14:41:35.495725 2733 tablet_service.cc:1514] Processing DeleteTablet for tablet e585b1c4774b4d398c9e801ad6f7ac92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:41:35 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:54378
I20260808 14:41:35.496143 2834 tablet_replica.cc:330] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607: stopping tablet replica
I20260808 14:41:35.496342 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 2371
I20260808 14:41:35.497393 2834 raft_consensus.cc:2229] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:41:35.497838 2834 raft_consensus.cc:2258] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:41:35.499754 2834 ts_tablet_manager.cc:1905] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:41:35.499786 2836 tablet_replica.cc:330] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c: stopping tablet replica
I20260808 14:41:35.500139 2836 raft_consensus.cc:2229] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:41:35.500556 2836 raft_consensus.cc:2258] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:41:35.503240 2834 ts_tablet_manager.cc:1918] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:41:35.503355 2834 log.cc:1196] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/e585b1c4774b4d398c9e801ad6f7ac92
I20260808 14:41:35.503727 2834 ts_tablet_manager.cc:1939] T e585b1c4774b4d398c9e801ad6f7ac92 P b8352d4aa8bc4e8fba90de4fd54ae607: Deleting consensus metadata
I20260808 14:41:35.505151 2318 catalog_manager.cc:4841] TS b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605): tablet e585b1c4774b4d398c9e801ad6f7ac92 (table test-table [id=6f62c775a3654d16ae5b51366db4aa3e]) successfully deleted
I20260808 14:41:35.506848 2836 ts_tablet_manager.cc:1905] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:41:35.509063 2836 ts_tablet_manager.cc:1918] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:41:35.509186 2836 log.cc:1196] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/e585b1c4774b4d398c9e801ad6f7ac92
I20260808 14:41:35.509564 2836 ts_tablet_manager.cc:1939] T e585b1c4774b4d398c9e801ad6f7ac92 P 7b934e85d4ac40f2b6adf6c68526024c: Deleting consensus metadata
I20260808 14:41:35.510987 2315 catalog_manager.cc:4841] TS 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67:39659): tablet e585b1c4774b4d398c9e801ad6f7ac92 (table test-table [id=6f62c775a3654d16ae5b51366db4aa3e]) successfully deleted
W20260808 14:41:35.511190 2318 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.144.65:36149 (error 108)
I20260808 14:41:35.511576 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 2526
I20260808 14:41:35.513792 2811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:35.513357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:36149 (local address 127.0.0.1:41884)
0808 14:41:35.513558 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:35.513675 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:36149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260808 14:41:35.513988 2318 catalog_manager.cc:4584] TS e383d29721074c8db3ba1c68ac20d57c (127.19.144.65:36149): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e585b1c4774b4d398c9e801ad6f7ac92: Network error: Client connection negotiation failed: client connection to 127.19.144.65:36149: connect: Connection refused (error 111)
I20260808 14:41:35.520886 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 2669
I20260808 14:41:35.527915 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 2298
I20260808 14:41:35.536586 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:43211
--webserver_interface=127.19.144.126
--webserver_port=34367
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:43211
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:41:35.654095 2838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:35.654417 2838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:35.654467 2838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:35.658783 2838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:41:35.658862 2838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:35.658883 2838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:35.658901 2838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:41:35.658916 2838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:41:35.664530 2838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:43211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:43211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=34367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:35.665823 2838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:35.667091 2838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:35.676296 2844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:35.676362 2838 server_base.cc:1034] running on GCE node
W20260808 14:41:35.676304 2843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:35.676309 2846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:35.677167 2838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:35.678390 2838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:35.679592 2838 hybrid_clock.cc:648] HybridClock initialized: now 1786200095679557 us; error 46 us; skew 500 ppm
Aug 08 14:41:35 dist-test-slave-p3t5 krb5kdc[1737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200095, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:35.682613 2838 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:41:35.683597 2838 webserver.cc:460] Webserver started at http://127.19.144.126:34367/ using document root <none> and password file <none>
I20260808 14:41:35.684209 2838 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:35.684314 2838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:35.688336 2838 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:41:35.690598 2853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:35.692094 2838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:41:35.692243 2838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "bf92c5ea5ec240578d9e6f8f4b227c26"
format_stamp: "Formatted at 2026-08-08 14:41:33 on dist-test-slave-p3t5"
server_key: "a4a80bb651ede6c22dc4f6e53d29a87f"
server_key_iv: "1cff0132a8395ea3a72b3e7b2d4828eb"
server_key_version: "encryptionkey@0"
I20260808 14:41:35.692811 2838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:35.703141 2838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:35.706831 2838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:35.707059 2838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:35.714571 2838 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:43211
I20260808 14:41:35.714881 2907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:43211 every 8 connection(s)
I20260808 14:41:35.715804 2838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:41:35.720871 2908 sys_catalog.cc:263] Verifying existing consensus state
I20260808 14:41:35.722992 2908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: Bootstrap starting.
I20260808 14:41:35.724505 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 2838
I20260808 14:41:35.725061 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:36149
--local_ip_for_outbound_sockets=127.19.144.65
--tserver_master_addrs=127.19.144.126:43211
--webserver_port=36391
--webserver_interface=127.19.144.65
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:41:35.744050 2908 log.cc:824] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:35.757781 2908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:41:35.758279 2908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26: Bootstrap complete.
I20260808 14:41:35.763170 2908 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } }
I20260808 14:41:35.763631 2908 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf92c5ea5ec240578d9e6f8f4b227c26, State: Initialized, Role: FOLLOWER
I20260808 14:41:35.764308 2908 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } }
I20260808 14:41:35.764458 2908 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:41:35.764566 2908 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:41:35.764704 2908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 2 FOLLOWER]: Advancing to term 3
I20260808 14:41:35.767143 2908 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } }
I20260808 14:41:35.767534 2908 leader_election.cc:302] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf92c5ea5ec240578d9e6f8f4b227c26; no voters:
I20260808 14:41:35.767938 2908 leader_election.cc:288] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 14:41:35.768043 2912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 14:41:35.768258 2912 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [term 3 LEADER]: Becoming Leader. State: Replica: bf92c5ea5ec240578d9e6f8f4b227c26, State: Running, Role: LEADER
I20260808 14:41:35.768603 2912 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } }
I20260808 14:41:35.769428 2908 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:41:35.769740 2912 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf92c5ea5ec240578d9e6f8f4b227c26. Latest consensus state: current_term: 3 leader_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } } }
I20260808 14:41:35.769872 2912 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [sys.catalog]: This master's current role is: LEADER
I20260808 14:41:35.770272 2913 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92c5ea5ec240578d9e6f8f4b227c26" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 43211 } } }
I20260808 14:41:35.770459 2913 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf92c5ea5ec240578d9e6f8f4b227c26 [sys.catalog]: This master's current role is: LEADER
I20260808 14:41:35.771718 2916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:41:35.777559 2916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:41:35.778409 2916 catalog_manager.cc:1259] Loaded cluster ID: b9c9317e9cd846149f0952d7300db951
I20260808 14:41:35.778486 2916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 14:41:35.779789 2916 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe631bc99ad google::LogMessage::Fail() at ??:0
@ 0x7fe631bcc88a google::LogMessage::SendToLog() at ??:0
@ 0x7fe631bc94de google::LogMessage::Flush() at ??:0
@ 0x7fe631bcaa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe63c508fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fe63c509d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe63c506579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fe63c54a640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe633e1b3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe633e1bc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fe633e1d3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe633e0c512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe6354506db start_thread at ??:0
@ 0x7fe6306c761f clone at ??:0
W20260808 14:41:35.883421 2910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:35.883723 2910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:35.883780 2910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:35.888077 2910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:35.888154 2910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:35.888216 2910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:41:35.893838 2910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:36149
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=36391
--tserver_master_addrs=127.19.144.126:43211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:35.895141 2910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:35.896270 2910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:35.906525 2952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:35.906524 2950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:35.906518 2949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:35.907367 2910 server_base.cc:1034] running on GCE node
I20260808 14:41:35.907835 2910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:35.908555 2910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:35.909739 2910 hybrid_clock.cc:648] HybridClock initialized: now 1786200095909695 us; error 52 us; skew 500 ppm
Aug 08 14:41:35 dist-test-slave-p3t5 krb5kdc[1737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200095, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:35.912969 2910 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:41:35.914055 2910 webserver.cc:460] Webserver started at http://127.19.144.65:36391/ using document root <none> and password file <none>
I20260808 14:41:35.914669 2910 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:35.914772 2910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:35.919183 2910 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:41:35.921805 2959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:35.923190 2910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:41:35.923343 2910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "e383d29721074c8db3ba1c68ac20d57c"
format_stamp: "Formatted at 2026-08-08 14:41:33 on dist-test-slave-p3t5"
server_key: "2e1e830b9634ec987cd2a84d56900712"
server_key_iv: "33c17947ca736e24b03300a8d816ce58"
server_key_version: "encryptionkey@0"
I20260808 14:41:35.923988 2910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:35.950759 2910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:35.954509 2910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:35.954738 2910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:35.955400 2910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:35.956734 2966 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 14:41:35.962849 2910 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 14:41:35.962950 2910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260808 14:41:35.963037 2910 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 14:41:35.964818 2910 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 14:41:35.964893 2910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 14:41:35.965199 2966 tablet_bootstrap.cc:492] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c: Bootstrap starting.
I20260808 14:41:35.976065 2910 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:36149
I20260808 14:41:35.976329 3076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:36149 every 8 connection(s)
I20260808 14:41:35.977181 2910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:41:35.984974 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 2910
I20260808 14:41:35.985477 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:35605
--local_ip_for_outbound_sockets=127.19.144.66
--tserver_master_addrs=127.19.144.126:43211
--webserver_port=45029
--webserver_interface=127.19.144.66
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:41:35.988057 2966 log.cc:824] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:35.996335 2966 tablet_bootstrap.cc:492] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:41:35.996747 2966 tablet_bootstrap.cc:492] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c: Bootstrap complete.
I20260808 14:41:35.997970 2966 ts_tablet_manager.cc:1397] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c: Time spent bootstrapping tablet: real 0.033s user 0.020s sys 0.011s
I20260808 14:41:36.000458 2966 raft_consensus.cc:348] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } }
I20260808 14:41:36.001080 2966 raft_consensus.cc:729] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e383d29721074c8db3ba1c68ac20d57c, State: Initialized, Role: FOLLOWER
I20260808 14:41:36.001632 2966 consensus_queue.cc:260] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } }
I20260808 14:41:36.002825 2966 ts_tablet_manager.cc:1428] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260808 14:41:36.009019 3079 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:41:35.979260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:42749)
0808 14:41:35.979765 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:35.979825 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:35.980691 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:36.007950 (+ 27259us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":145,"threads_started":1}
W20260808 14:41:36.009667 3077 heartbeater.cc:644] Failed to heartbeat to 127.19.144.126:43211 (0 consecutive failures): Network error: Failed to ping master at 127.19.144.126:43211: Client connection negotiation failed: client connection to 127.19.144.126:43211: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 14:41:36.010452 3079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:36.010155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:34015)
0808 14:41:36.010309 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:36.010384 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 14:41:36.011143 3079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:36.010913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:38175)
0808 14:41:36.011027 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:36.011078 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 14:41:36.011335 3077 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 14:41:36.114830 3081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:36.115231 3081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:36.115339 3081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:36.119549 3081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:36.119656 3081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:36.119755 3081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:41:36.125288 3081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:35605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=45029
--tserver_master_addrs=127.19.144.126:43211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:36.126456 3081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:36.127583 3081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:36.138253 3087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:36.138397 3090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:36.138278 3088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:36.138578 3081 server_base.cc:1034] running on GCE node
I20260808 14:41:36.139048 3081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:36.139842 3081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:36.141024 3081 hybrid_clock.cc:648] HybridClock initialized: now 1786200096141006 us; error 30 us; skew 500 ppm
Aug 08 14:41:36 dist-test-slave-p3t5 krb5kdc[1737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200096, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:36.143975 3081 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:41:36.144945 3081 webserver.cc:460] Webserver started at http://127.19.144.66:45029/ using document root <none> and password file <none>
I20260808 14:41:36.145581 3081 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:36.145728 3081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:36.149931 3081 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 14:41:36.152231 3097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:36.153697 3081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:41:36.153856 3081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "b8352d4aa8bc4e8fba90de4fd54ae607"
format_stamp: "Formatted at 2026-08-08 14:41:33 on dist-test-slave-p3t5"
server_key: "b9da2594f4341d8a39eaf6f317e9588c"
server_key_iv: "53ffacb03f8f93be1e604fd432559d79"
server_key_version: "encryptionkey@0"
I20260808 14:41:36.154458 3081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:36.168390 3081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:36.172156 3081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:36.172400 3081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:36.173089 3081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:36.174067 3081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:41:36.174145 3081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:36.174223 3081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:41:36.174269 3081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:36.183971 3081 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:35605
I20260808 14:41:36.184000 3214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:35605 every 8 connection(s)
I20260808 14:41:36.185113 3081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:41:36.186265 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 3081
I20260808 14:41:36.186704 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:39659
--local_ip_for_outbound_sockets=127.19.144.67
--tserver_master_addrs=127.19.144.126:43211
--webserver_port=38335
--webserver_interface=127.19.144.67
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:41:36.189772 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:36.187572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:51427)
0808 14:41:36.188119 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:36.188264 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":148,"threads_started":1}
W20260808 14:41:36.190397 3215 heartbeater.cc:644] Failed to heartbeat to 127.19.144.126:43211 (0 consecutive failures): Network error: Failed to ping master at 127.19.144.126:43211: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
I20260808 14:41:36.191164 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:36.190911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:34931)
0808 14:41:36.191045 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:36.191101 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 14:41:36.191936 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:36.191632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:58559)
0808 14:41:36.191775 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:36.191844 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 14:41:36.192139 3215 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 14:41:36.308676 3218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:36.309026 3218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:36.309074 3218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:36.313339 3218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:36.313433 3218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:36.313495 3218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:41:36.319308 3218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:39659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=38335
--tserver_master_addrs=127.19.144.126:43211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:36.320516 3218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:36.321676 3218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:36.332118 3223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:36.332202 3226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:36.332341 3218 server_base.cc:1034] running on GCE node
W20260808 14:41:36.332209 3224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:36.332896 3218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:36.333606 3218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:36.334846 3218 hybrid_clock.cc:648] HybridClock initialized: now 1786200096334806 us; error 52 us; skew 500 ppm
Aug 08 14:41:36 dist-test-slave-p3t5 krb5kdc[1737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200096, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:36.337677 3218 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:41:36.338579 3218 webserver.cc:460] Webserver started at http://127.19.144.67:38335/ using document root <none> and password file <none>
I20260808 14:41:36.339172 3218 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:36.339282 3218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:36.343170 3218 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:41:36.345397 3233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:36.346784 3218 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:41:36.346936 3218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "7b934e85d4ac40f2b6adf6c68526024c"
format_stamp: "Formatted at 2026-08-08 14:41:34 on dist-test-slave-p3t5"
server_key: "5ceedeae0ac68b7836644c83bec9cc59"
server_key_iv: "3e0592cf1e3b2427c3fd5c449aac45a9"
server_key_version: "encryptionkey@0"
I20260808 14:41:36.347548 3218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:36.361351 3218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:36.365494 3218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:36.365809 3218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:36.366524 3218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:36.367504 3218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:41:36.367559 3218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:36.367640 3218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:41:36.367682 3218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:36.377821 3218 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:39659
I20260808 14:41:36.377844 3349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:39659 every 8 connection(s)
I20260808 14:41:36.379060 3218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:41:36.383594 3352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:36.381405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:58593)
0808 14:41:36.382001 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:36.382143 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":185,"threads_started":1}
W20260808 14:41:36.384145 3350 heartbeater.cc:644] Failed to heartbeat to 127.19.144.126:43211 (0 consecutive failures): Network error: Failed to ping master at 127.19.144.126:43211: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
I20260808 14:41:36.384835 3352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:36.384591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:33441)
0808 14:41:36.384710 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:36.384767 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 14:41:36.385510 3352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:36.385280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:41727)
0808 14:41:36.385399 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:36.385458 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 14:41:36.385743 3350 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 14:41:36.386242 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 3218
I20260808 14:41:37.012564 3354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:37.012059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:39747)
0808 14:41:37.012388 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:37.012455 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":90,"threads_started":1}
I20260808 14:41:37.193349 3355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:37.192862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:40157)
0808 14:41:37.193163 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:37.193233 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":81,"threads_started":1}
I20260808 14:41:37.386752 3356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:37.386316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:47055)
0808 14:41:37.386575 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:37.386631 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":92,"threads_started":1}
I20260808 14:41:37.791507 3357 raft_consensus.cc:482] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:41:37.791762 3357 raft_consensus.cc:504] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } }
I20260808 14:41:37.793008 3357 leader_election.cc:288] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605), 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67:39659)
Aug 08 14:41:37 dist-test-slave-p3t5 krb5kdc[1737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200095, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.67@KRBTEST.COM
Aug 08 14:41:37 dist-test-slave-p3t5 krb5kdc[1737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200095, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.66@KRBTEST.COM
I20260808 14:41:37.808007 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:37.793283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:39659 (local address 127.19.144.65:57441)
0808 14:41:37.793591 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:37.793605 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:37.793772 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:37.795434 (+ 1662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:37.795441 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:37.796043 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:37.796518 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:37.796535 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:37.797953 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:37.797961 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:37.798884 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:37.798894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:37.799696 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:37.799699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:37.799841 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:37.800577 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:37.800593 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:37.802406 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:37.805240 (+ 2834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:37.805247 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:37.805264 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:37.805587 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:37.806024 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:37.806027 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:37.806029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:37.806187 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:37.806904 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:37.806911 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:37.807186 (+ 275us) client_negotiation.cc:769] Sending connection context
0808 14:41:37.807382 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 14:41:37.807783 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":157,"spinlock_wait_cycles":6272,"thread_start_us":80,"threads_started":1}
I20260808 14:41:37.808007 3359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:37.793434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:57441 (local address 127.19.144.67:39659)
0808 14:41:37.793887 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:37.793921 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:37.793964 (+ 43us) server_negotiation.cc:407] Connection header received
0808 14:41:37.794687 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:37.794693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:37.795054 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:37.795382 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:37.796711 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:37.797802 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:37.799044 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:37.799535 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:37.799745 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:37.802668 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:37.802673 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:37.802687 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:37.802718 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:37.805103 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:37.805750 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:37.805757 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:37.805763 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:37.805905 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:37.806296 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:37.806300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:37.806302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:37.806778 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:37.806932 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:37.807527 (+ 595us) server_negotiation.cc:299] Negotiation successful
0808 14:41:37.807801 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":76,"threads_started":1}
I20260808 14:41:37.808023 3360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:37.793542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:59379 (local address 127.19.144.66:35605)
0808 14:41:37.793979 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:37.793995 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:37.794274 (+ 279us) server_negotiation.cc:407] Connection header received
0808 14:41:37.794994 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:37.794999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:37.795296 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:37.795590 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:37.796707 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:37.797802 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:37.799026 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:37.799518 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:37.799706 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:37.802956 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:37.802963 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:37.802975 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:37.803007 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:37.805259 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:37.805855 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:37.805860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:37.805865 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:37.805985 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:37.806314 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:37.806317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:37.806318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:37.806777 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:37.806931 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:37.807523 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 14:41:37.807818 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":110,"threads_started":1}
I20260808 14:41:37.808063 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:37.793285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:35605 (local address 127.19.144.65:59379)
0808 14:41:37.794144 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:37.794156 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:37.794290 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:37.795537 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:37.795540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:37.796060 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:37.796519 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:37.796536 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:37.797949 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:37.797953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:37.798883 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:37.798893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:37.799658 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:37.799661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:37.799841 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:37.800630 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:37.800643 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:37.802760 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:37.805394 (+ 2634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:37.805400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:37.805404 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:37.805722 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:37.806087 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:37.806091 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:37.806093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:37.806200 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:37.806903 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:37.806911 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:37.807186 (+ 275us) client_negotiation.cc:769] Sending connection context
0808 14:41:37.807382 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 14:41:37.807872 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":735,"spinlock_wait_cycles":7808,"thread_start_us":79,"threads_started":1}
W20260808 14:41:37.810993 2960 leader_election.cc:341] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67:39659): Not found: Tablet not found: e585b1c4774b4d398c9e801ad6f7ac92
W20260808 14:41:37.811167 2963 leader_election.cc:341] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605): Not found: Tablet not found: e585b1c4774b4d398c9e801ad6f7ac92
I20260808 14:41:37.811276 2963 leader_election.cc:302] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e383d29721074c8db3ba1c68ac20d57c; no voters: 7b934e85d4ac40f2b6adf6c68526024c, b8352d4aa8bc4e8fba90de4fd54ae607
I20260808 14:41:37.811589 3357 raft_consensus.cc:2726] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:41:38.014694 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:38.014015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:52625)
0808 14:41:38.014322 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:38.014450 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260808 14:41:38.195499 3362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:38.194504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:41651)
0808 14:41:38.195072 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:38.195181 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":163,"threads_started":1}
I20260808 14:41:38.388105 3363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:38.387632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:44343)
0808 14:41:38.387934 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:38.388004 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":81,"threads_started":1}
I20260808 14:41:39.016451 3364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:39.015925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:43917)
0808 14:41:39.016255 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:39.016324 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":92,"threads_started":1}
I20260808 14:41:39.197157 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:39.196662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:59091)
0808 14:41:39.196978 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:39.197047 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":86,"threads_started":1}
I20260808 14:41:39.389737 3366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:39.389122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:57785)
0808 14:41:39.389457 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:39.389534 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":95,"threads_started":1}
I20260808 14:41:39.460994 3367 raft_consensus.cc:482] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:41:39.461146 3367 raft_consensus.cc:504] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } }
I20260808 14:41:39.461644 3367 leader_election.cc:288] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605), 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67:39659)
W20260808 14:41:39.462568 2963 leader_election.cc:341] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605): Not found: Tablet not found: e585b1c4774b4d398c9e801ad6f7ac92
W20260808 14:41:39.462674 2960 leader_election.cc:341] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67:39659): Not found: Tablet not found: e585b1c4774b4d398c9e801ad6f7ac92
I20260808 14:41:39.462802 2960 leader_election.cc:302] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e383d29721074c8db3ba1c68ac20d57c; no voters: 7b934e85d4ac40f2b6adf6c68526024c, b8352d4aa8bc4e8fba90de4fd54ae607
I20260808 14:41:39.462956 3367 raft_consensus.cc:2726] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:41:40.017992 3368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:40.017313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:36169)
0808 14:41:40.017718 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:40.017870 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":121,"threads_started":1}
I20260808 14:41:40.198839 3369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:40.198219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:53765)
0808 14:41:40.198524 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:40.198593 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":88,"threads_started":1}
I20260808 14:41:40.391098 3370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:40.390635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:52657)
0808 14:41:40.390930 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:40.390995 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":87,"threads_started":1}
I20260808 14:41:41.019595 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:41.019056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:46931)
0808 14:41:41.019413 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:41.019481 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":81,"threads_started":1}
I20260808 14:41:41.200474 3372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:41.199939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:51897)
0808 14:41:41.200271 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:41.200337 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":90,"threads_started":1}
I20260808 14:41:41.392382 3373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:41.391930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:54905)
0808 14:41:41.392220 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:41.392280 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":100,"threads_started":1}
I20260808 14:41:42.021178 3374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:42.020680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:34493)
0808 14:41:42.021002 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:42.021072 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":96,"threads_started":1}
I20260808 14:41:42.201835 3375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:42.201285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:37555)
0808 14:41:42.201589 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:42.201720 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":86,"threads_started":1}
I20260808 14:41:42.393847 3376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:42.393272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:40971)
0808 14:41:42.393600 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:42.393737 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":102,"threads_started":1}
I20260808 14:41:42.724931 3377 raft_consensus.cc:482] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:41:42.725085 3377 raft_consensus.cc:504] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } }
I20260808 14:41:42.725665 3377 leader_election.cc:288] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605), 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67:39659)
W20260808 14:41:42.726595 2960 leader_election.cc:341] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67:39659): Not found: Tablet not found: e585b1c4774b4d398c9e801ad6f7ac92
W20260808 14:41:42.726765 2963 leader_election.cc:341] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605): Not found: Tablet not found: e585b1c4774b4d398c9e801ad6f7ac92
I20260808 14:41:42.726866 2963 leader_election.cc:302] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e383d29721074c8db3ba1c68ac20d57c; no voters: 7b934e85d4ac40f2b6adf6c68526024c, b8352d4aa8bc4e8fba90de4fd54ae607
I20260808 14:41:42.727074 3377 raft_consensus.cc:2726] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:41:43.022797 3378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:43.022167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:35573)
0808 14:41:43.022559 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:43.022635 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":114,"threads_started":1}
I20260808 14:41:43.203315 3379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:43.202706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:42025)
0808 14:41:43.203058 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:43.203151 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":87,"threads_started":1}
I20260808 14:41:43.395397 3380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:43.394770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:45101)
0808 14:41:43.395214 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:43.395289 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":138,"threads_started":1}
I20260808 14:41:44.024294 3382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:44.023576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:34323)
0808 14:41:44.023918 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:44.023981 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":97,"threads_started":1}
I20260808 14:41:44.204663 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:44.204185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:38813)
0808 14:41:44.204497 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:44.204558 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":118,"threads_started":1}
I20260808 14:41:44.396874 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:44.396411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:38371)
0808 14:41:44.396706 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:44.396771 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":90,"threads_started":1}
I20260808 14:41:45.025846 3385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:45.025238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:42435)
0808 14:41:45.025591 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:45.025742 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":110,"threads_started":1}
I20260808 14:41:45.205996 3386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:45.205517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:54435)
0808 14:41:45.205830 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:45.205890 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":152,"threads_started":1}
I20260808 14:41:45.398325 3387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:45.397836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:37497)
0808 14:41:45.398157 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:45.398223 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":116,"threads_started":1}
I20260808 14:41:45.613581 3388 raft_consensus.cc:482] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:41:45.613855 3388 raft_consensus.cc:504] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } }
I20260808 14:41:45.614358 3388 leader_election.cc:288] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605), 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67:39659)
W20260808 14:41:45.615315 2960 leader_election.cc:341] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67:39659): Not found: Tablet not found: e585b1c4774b4d398c9e801ad6f7ac92
W20260808 14:41:45.615432 2963 leader_election.cc:341] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605): Not found: Tablet not found: e585b1c4774b4d398c9e801ad6f7ac92
I20260808 14:41:45.615554 2963 leader_election.cc:302] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e383d29721074c8db3ba1c68ac20d57c; no voters: 7b934e85d4ac40f2b6adf6c68526024c, b8352d4aa8bc4e8fba90de4fd54ae607
I20260808 14:41:45.615706 3388 raft_consensus.cc:2726] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:41:46.027470 3389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:46.026985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:53219)
0808 14:41:46.027276 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:46.027342 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":101,"threads_started":1}
I20260808 14:41:46.207370 3391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:46.206893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:58145)
0808 14:41:46.207186 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:46.207261 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":100,"threads_started":1}
I20260808 14:41:46.399724 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:46.399272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:44661)
0808 14:41:46.399557 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:46.399622 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":77,"threads_started":1}
I20260808 14:41:47.028847 3393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:47.028383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:38381)
0808 14:41:47.028673 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:47.028738 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":106,"threads_started":1}
I20260808 14:41:47.208623 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:47.208191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:44417)
0808 14:41:47.208459 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:47.208516 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":83,"threads_started":1}
I20260808 14:41:47.401095 3395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:47.400613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:42657)
0808 14:41:47.400924 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:47.400990 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":78,"threads_started":1}
I20260808 14:41:48.017678 3396 raft_consensus.cc:482] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:41:48.017845 3396 raft_consensus.cc:504] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8352d4aa8bc4e8fba90de4fd54ae607" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 35605 } } peers { permanent_uuid: "7b934e85d4ac40f2b6adf6c68526024c" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39659 } } peers { permanent_uuid: "e383d29721074c8db3ba1c68ac20d57c" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 36149 } }
I20260808 14:41:48.018332 3396 leader_election.cc:288] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605), 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67:39659)
W20260808 14:41:48.019191 2960 leader_election.cc:341] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b934e85d4ac40f2b6adf6c68526024c (127.19.144.67:39659): Not found: Tablet not found: e585b1c4774b4d398c9e801ad6f7ac92
W20260808 14:41:48.019323 2963 leader_election.cc:341] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8352d4aa8bc4e8fba90de4fd54ae607 (127.19.144.66:35605): Not found: Tablet not found: e585b1c4774b4d398c9e801ad6f7ac92
I20260808 14:41:48.019419 2963 leader_election.cc:302] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e383d29721074c8db3ba1c68ac20d57c; no voters: 7b934e85d4ac40f2b6adf6c68526024c, b8352d4aa8bc4e8fba90de4fd54ae607
I20260808 14:41:48.019594 3396 raft_consensus.cc:2726] T e585b1c4774b4d398c9e801ad6f7ac92 P e383d29721074c8db3ba1c68ac20d57c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:41:48.030435 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:48.029865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:37879)
0808 14:41:48.030190 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:48.030295 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":116,"threads_started":1}
I20260808 14:41:48.210022 3398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:48.209418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:42609)
0808 14:41:48.209784 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:48.209885 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":87,"threads_started":1}
I20260808 14:41:48.402608 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:48.402136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:40823)
0808 14:41:48.402436 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:48.402503 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":81,"threads_started":1}
I20260808 14:41:49.031812 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:49.031347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:48331)
0808 14:41:49.031641 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:49.031705 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":87,"threads_started":1}
I20260808 14:41:49.211400 3401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:49.210919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:54727)
0808 14:41:49.211221 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:49.211290 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":87,"threads_started":1}
I20260808 14:41:49.404203 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:49.403592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:53517)
0808 14:41:49.404033 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:49.404100 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":232,"threads_started":1}
I20260808 14:41:50.033195 3403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:50.032700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:41833)
0808 14:41:50.033022 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:50.033090 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":137,"threads_started":1}
I20260808 14:41:50.212889 3404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:50.212373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:35819)
0808 14:41:50.212710 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:50.212776 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":121,"threads_started":1}
I20260808 14:41:50.405719 3405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:50.405131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:39435)
0808 14:41:50.405458 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:50.405529 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":111,"threads_started":1}
I20260808 14:41:51.034819 3406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:51.034351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.65:38379)
0808 14:41:51.034648 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:51.034714 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":106,"threads_started":1}
I20260808 14:41:51.214519 3407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:51.214058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.66:49085)
0808 14:41:51.214345 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:51.214411 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":86,"threads_started":1}
I20260808 14:41:51.395129 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 2910
I20260808 14:41:51.404682 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 3081
I20260808 14:41:51.407550 3408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:51.407112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:43211 (local address 127.19.144.67:47727)
0808 14:41:51.407393 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:51.407454 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:43211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":85,"threads_started":1}
I20260808 14:41:51.412534 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 3218
2026-08-08T14:41:51Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19511 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:41:51 dist-test-slave-p3t5 krb5kdc[3411](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:41:51 dist-test-slave-p3t5 krb5kdc[3411](info): set up 2 sockets
Aug 08 14:41:51 dist-test-slave-p3t5 krb5kdc[3411](info): commencing operation
krb5kdc: starting...
W20260808 14:41:52.495903 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:41:52 dist-test-slave-p3t5 krb5kdc[3411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:41:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:41:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:41:52.640513 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:35049
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:35049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:41:52.762187 3426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:52.762590 3426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:52.762679 3426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:52.767011 3426 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 14:41:52.767136 3426 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 14:41:52.767166 3426 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 14:41:52.767189 3426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:41:52.767235 3426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:52.767272 3426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:52.767313 3426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:41:52.767364 3426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:41:52.773208 3426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:35049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:35049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:52.774607 3426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:52.775787 3426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:52.786190 3432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:52.786168 3431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:52.786168 3434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:52.786238 3426 server_base.cc:1034] running on GCE node
I20260808 14:41:52.787245 3426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:52.788570 3426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:52.789733 3426 hybrid_clock.cc:648] HybridClock initialized: now 1786200112789711 us; error 38 us; skew 500 ppm
Aug 08 14:41:52 dist-test-slave-p3t5 krb5kdc[3411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200112, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:52.792698 3426 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:41:52.793776 3426 webserver.cc:460] Webserver started at http://127.19.144.126:43281/ using document root <none> and password file <none>
I20260808 14:41:52.794412 3426 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:52.794515 3426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:52.794777 3426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:41:52.796993 3426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "0d45fad648bc448893b80c7429c4ee52"
format_stamp: "Formatted at 2026-08-08 14:41:52 on dist-test-slave-p3t5"
server_key: "2f39c340e0d9bab58502875f05274937"
server_key_iv: "3a7d9de9ee07a7932c356b961296a232"
server_key_version: "encryptionkey@0"
I20260808 14:41:52.797660 3426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "0d45fad648bc448893b80c7429c4ee52"
format_stamp: "Formatted at 2026-08-08 14:41:52 on dist-test-slave-p3t5"
server_key: "2f39c340e0d9bab58502875f05274937"
server_key_iv: "3a7d9de9ee07a7932c356b961296a232"
server_key_version: "encryptionkey@0"
I20260808 14:41:52.801708 3426 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:41:52.804252 3441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:52.805552 3426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:41:52.805740 3426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "0d45fad648bc448893b80c7429c4ee52"
format_stamp: "Formatted at 2026-08-08 14:41:52 on dist-test-slave-p3t5"
server_key: "2f39c340e0d9bab58502875f05274937"
server_key_iv: "3a7d9de9ee07a7932c356b961296a232"
server_key_version: "encryptionkey@0"
I20260808 14:41:52.805874 3426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:52.830960 3426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:52.834440 3426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:52.834705 3426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:52.842478 3426 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:35049
I20260808 14:41:52.842814 3495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:35049 every 8 connection(s)
I20260808 14:41:52.843554 3426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:41:52.846602 3496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:52.849082 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 3426
I20260808 14:41:52.849217 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:41:52.849511 20033 external_mini_cluster.cc:1428] Setting key 0513e96acaf3909faf28ad752f0d631d
I20260808 14:41:52.852913 3496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Bootstrap starting.
I20260808 14:41:52.855719 3496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:52.856600 3496 log.cc:824] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:52.859238 3496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: No bootstrap required, opened a new log
Aug 08 14:41:52 dist-test-slave-p3t5 krb5kdc[3411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:52.863292 3496 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } }
I20260808 14:41:52.863580 3496 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:52.863720 3496 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d45fad648bc448893b80c7429c4ee52, State: Initialized, Role: FOLLOWER
I20260808 14:41:52.864318 3496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } }
I20260808 14:41:52.864488 3496 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:41:52.864560 3496 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:41:52.864681 3496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:52.865306 3499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:52.851105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59880 (local address 127.19.144.126:35049)
0808 14:41:52.851553 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:52.851569 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:52.851607 (+ 38us) server_negotiation.cc:407] Connection header received
0808 14:41:52.852262 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:52.852280 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:52.852640 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:52.852935 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:52.853966 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:52.854970 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:52.856172 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:52.856665 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:52.856895 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:52.859934 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:52.859940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:52.859955 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:52.859989 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:52.861856 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:52.862368 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:52.862373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:52.862379 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:52.862513 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:52.862828 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:52.862831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:52.862833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:52.863290 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:52.863540 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:52.863849 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 14:41:52.864182 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":142,"threads_started":1}
I20260808 14:41:52.866001 3496 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } }
I20260808 14:41:52.866384 3496 leader_election.cc:302] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d45fad648bc448893b80c7429c4ee52; no voters:
I20260808 14:41:52.866815 3496 leader_election.cc:288] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:41:52.866891 3501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:41:52.867101 3501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 1 LEADER]: Becoming Leader. State: Replica: 0d45fad648bc448893b80c7429c4ee52, State: Running, Role: LEADER
I20260808 14:41:52.867486 3501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } }
I20260808 14:41:52.868209 3496 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:41:52.868780 3501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d45fad648bc448893b80c7429c4ee52. Latest consensus state: current_term: 1 leader_uuid: "0d45fad648bc448893b80c7429c4ee52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } } }
I20260808 14:41:52.868997 3501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [sys.catalog]: This master's current role is: LEADER
I20260808 14:41:52.869230 3502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d45fad648bc448893b80c7429c4ee52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } } }
I20260808 14:41:52.869382 3502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [sys.catalog]: This master's current role is: LEADER
I20260808 14:41:52.869803 3508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:41:52.873116 3508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:41:52.879456 3508 catalog_manager.cc:1347] Generated new cluster ID: 1b214b6b360142ca8c5cec785f370806
I20260808 14:41:52.879547 3508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:41:52.899932 3508 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:41:52.901401 3508 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:41:52.908205 3508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Generated new TSK 0
I20260808 14:41:52.909061 3508 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:41:52.945214 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35049
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:41:53.070230 3522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:53.070576 3522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:53.070660 3522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:53.074753 3522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:53.074860 3522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:53.075002 3522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:41:53.080423 3522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:53.081648 3522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:53.082768 3522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:53.093312 3530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:53.093458 3522 server_base.cc:1034] running on GCE node
W20260808 14:41:53.093327 3527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:53.093327 3528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:53.094113 3522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:53.094820 3522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:53.095988 3522 hybrid_clock.cc:648] HybridClock initialized: now 1786200113095973 us; error 27 us; skew 500 ppm
Aug 08 14:41:53 dist-test-slave-p3t5 krb5kdc[3411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200113, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:53.099261 3522 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:41:53.100355 3522 webserver.cc:460] Webserver started at http://127.19.144.65:44069/ using document root <none> and password file <none>
I20260808 14:41:53.100981 3522 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:53.101058 3522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:53.101320 3522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:41:53.103710 3522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "0e1e723868bc4464832909357429ad27"
format_stamp: "Formatted at 2026-08-08 14:41:53 on dist-test-slave-p3t5"
server_key: "8c0fafcd93394c51ec44b13fcabd6597"
server_key_iv: "400ab80e369ded9b435fc00c96eaa1f5"
server_key_version: "encryptionkey@0"
I20260808 14:41:53.104341 3522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "0e1e723868bc4464832909357429ad27"
format_stamp: "Formatted at 2026-08-08 14:41:53 on dist-test-slave-p3t5"
server_key: "8c0fafcd93394c51ec44b13fcabd6597"
server_key_iv: "400ab80e369ded9b435fc00c96eaa1f5"
server_key_version: "encryptionkey@0"
I20260808 14:41:53.108541 3522 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 14:41:53.111500 3537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:53.112829 3522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:41:53.113029 3522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "0e1e723868bc4464832909357429ad27"
format_stamp: "Formatted at 2026-08-08 14:41:53 on dist-test-slave-p3t5"
server_key: "8c0fafcd93394c51ec44b13fcabd6597"
server_key_iv: "400ab80e369ded9b435fc00c96eaa1f5"
server_key_version: "encryptionkey@0"
I20260808 14:41:53.113147 3522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:53.150317 3522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:53.154119 3522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:53.154296 3522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:53.154896 3522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:53.155761 3522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:41:53.155812 3522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:53.155854 3522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:41:53.155869 3522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:53.165380 3522 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:41059
I20260808 14:41:53.165411 3653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:41059 every 8 connection(s)
I20260808 14:41:53.166479 3522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:41:53.174098 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 3522
I20260808 14:41:53.174224 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:41:53.174532 20033 external_mini_cluster.cc:1428] Setting key a62585e7b913667bc66e9b15e0974fbd
Aug 08 14:41:53 dist-test-slave-p3t5 krb5kdc[3411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200113, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:53.181609 3499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.168470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:43101 (local address 127.19.144.126:35049)
0808 14:41:53.168665 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:53.168670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:53.169676 (+ 1006us) server_negotiation.cc:407] Connection header received
0808 14:41:53.171138 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:53.171141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:53.171193 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:53.171271 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:53.172546 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.173084 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.174375 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.174645 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.174779 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.177694 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:53.177698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:53.177700 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:53.177725 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:53.179346 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:53.179911 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:53.179915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:53.179917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:53.180019 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:53.180394 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:53.180397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:53.180399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:53.180608 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:53.180722 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:53.181284 (+ 562us) server_negotiation.cc:299] Negotiation successful
0808 14:41:53.181433 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 14:41:53.182770 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.168974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:43101)
0808 14:41:53.169468 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:53.169518 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:53.170920 (+ 1402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:53.171488 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:53.171495 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:53.171884 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:53.172395 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:53.172410 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.173199 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.173202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:53.174226 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:53.174241 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.174828 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.174832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:53.175066 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.175799 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:53.175845 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:53.177485 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:53.179486 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:53.179492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:53.179509 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:53.179798 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:53.180120 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:53.180123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:53.180126 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:53.180289 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:53.180723 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:53.180733 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:53.181019 (+ 286us) client_negotiation.cc:769] Sending connection context
0808 14:41:53.181226 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 14:41:53.181605 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":123,"threads_started":1}
I20260808 14:41:53.184235 3654 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35049
I20260808 14:41:53.184559 3654 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:53.185243 3654 heartbeater.cc:511] Master 127.19.144.126:35049 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:41:53.187222 3458 ts_manager.cc:194] Registered new tserver with Master: 0e1e723868bc4464832909357429ad27 (127.19.144.65:41059)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
I20260808 14:41:53.188509 3458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:43101
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:41:53.207258 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35049
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:41:53.355969 3659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:53.356312 3659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:53.356397 3659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:53.360708 3659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:53.360827 3659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:53.360922 3659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:41:53.366683 3659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:53.367928 3659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:53.369231 3659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:53.379721 3664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:53.379853 3667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:53.379921 3659 server_base.cc:1034] running on GCE node
W20260808 14:41:53.380210 3665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:53.380681 3659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:53.381389 3659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:53.382587 3659 hybrid_clock.cc:648] HybridClock initialized: now 1786200113382557 us; error 74 us; skew 500 ppm
Aug 08 14:41:53 dist-test-slave-p3t5 krb5kdc[3411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200113, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:53.385696 3659 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:41:53.386794 3659 webserver.cc:460] Webserver started at http://127.19.144.66:45287/ using document root <none> and password file <none>
I20260808 14:41:53.387421 3659 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:53.387519 3659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:53.387749 3659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:41:53.390044 3659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "1466ce81810245f099fcf8dfa47444d2"
format_stamp: "Formatted at 2026-08-08 14:41:53 on dist-test-slave-p3t5"
server_key: "c188e8471ab5d02c4986db89502f2be6"
server_key_iv: "c0806f8b9eac0b4c34716cef43e51b39"
server_key_version: "encryptionkey@0"
I20260808 14:41:53.390655 3659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "1466ce81810245f099fcf8dfa47444d2"
format_stamp: "Formatted at 2026-08-08 14:41:53 on dist-test-slave-p3t5"
server_key: "c188e8471ab5d02c4986db89502f2be6"
server_key_iv: "c0806f8b9eac0b4c34716cef43e51b39"
server_key_version: "encryptionkey@0"
I20260808 14:41:53.394833 3659 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:41:53.397567 3674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:53.399063 3659 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:41:53.399231 3659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "1466ce81810245f099fcf8dfa47444d2"
format_stamp: "Formatted at 2026-08-08 14:41:53 on dist-test-slave-p3t5"
server_key: "c188e8471ab5d02c4986db89502f2be6"
server_key_iv: "c0806f8b9eac0b4c34716cef43e51b39"
server_key_version: "encryptionkey@0"
I20260808 14:41:53.399371 3659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:53.417166 3659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:53.420599 3659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:53.420835 3659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:53.421499 3659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:53.422441 3659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:41:53.422490 3659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:53.422561 3659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:41:53.422598 3659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:53.431972 3659 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:34323
I20260808 14:41:53.431991 3788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:34323 every 8 connection(s)
I20260808 14:41:53.433055 3659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:41:53.436594 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 3659
I20260808 14:41:53.436705 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:41:53.436992 20033 external_mini_cluster.cc:1428] Setting key eba2c26d309ffa0663acf1a37a0501cc
Aug 08 14:41:53 dist-test-slave-p3t5 krb5kdc[3411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200113, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
I20260808 14:41:53.448421 3499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.435010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:38569 (local address 127.19.144.126:35049)
0808 14:41:53.435166 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:53.435170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:53.436080 (+ 910us) server_negotiation.cc:407] Connection header received
0808 14:41:53.437651 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:53.437658 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:53.437723 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:53.437837 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:53.439230 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.440079 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.441269 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.441528 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.441670 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.444337 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:53.444341 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:53.444344 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:53.444376 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:53.446044 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:53.446590 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:53.446594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:53.446596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:53.446690 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:53.447079 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:53.447084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:53.447085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:53.447313 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:53.447428 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:53.448081 (+ 653us) server_negotiation.cc:299] Negotiation successful
0808 14:41:53.448234 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:41:53.449491 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.435429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:38569)
0808 14:41:53.435893 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:53.435938 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:53.437365 (+ 1427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:53.438026 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:53.438033 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:53.438542 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:53.439071 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:53.439088 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.440214 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.440217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:53.441126 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:53.441136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.441721 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.441726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:53.441903 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.442610 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:53.442628 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:53.444159 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:53.446171 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:53.446178 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:53.446196 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:53.446478 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:53.446804 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:53.446808 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:53.446810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:53.446974 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:53.447430 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:53.447439 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:53.447759 (+ 320us) client_negotiation.cc:769] Sending connection context
0808 14:41:53.448018 (+ 259us) client_negotiation.cc:240] Negotiation successful
0808 14:41:53.448386 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":115,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
I20260808 14:41:53.450825 3789 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35049
I20260808 14:41:53.451112 3789 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:53.451660 3789 heartbeater.cc:511] Master 127.19.144.126:35049 requested a full tablet report, sending...
I20260808 14:41:53.452867 3458 ts_manager.cc:194] Registered new tserver with Master: 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66:34323)
I20260808 14:41:53.453501 3458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:38569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:41:53.469578 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35049
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:41:53.589433 3794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:41:53.589810 3794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:41:53.589897 3794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:41:53.594029 3794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:41:53.594095 3794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:41:53.594208 3794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:41:53.599852 3794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:41:53.601051 3794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:41:53.602317 3794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:41:53.613709 3800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:53.613709 3802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:41:53.613781 3799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:41:53.613898 3794 server_base.cc:1034] running on GCE node
I20260808 14:41:53.614563 3794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:41:53.615239 3794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:41:53.616437 3794 hybrid_clock.cc:648] HybridClock initialized: now 1786200113616422 us; error 29 us; skew 500 ppm
Aug 08 14:41:53 dist-test-slave-p3t5 krb5kdc[3411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200113, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:41:53.619511 3794 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:41:53.620407 3794 webserver.cc:460] Webserver started at http://127.19.144.67:33425/ using document root <none> and password file <none>
I20260808 14:41:53.620972 3794 fs_manager.cc:359] Metadata directory not provided
I20260808 14:41:53.621037 3794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:41:53.621217 3794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:41:53.623461 3794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "a93271875de240889bb59007953575ce"
format_stamp: "Formatted at 2026-08-08 14:41:53 on dist-test-slave-p3t5"
server_key: "d1dd45421e15d389754beff2fd012c23"
server_key_iv: "7fb0f886192448f646d72c137cbdb23f"
server_key_version: "encryptionkey@0"
I20260808 14:41:53.624019 3794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "a93271875de240889bb59007953575ce"
format_stamp: "Formatted at 2026-08-08 14:41:53 on dist-test-slave-p3t5"
server_key: "d1dd45421e15d389754beff2fd012c23"
server_key_iv: "7fb0f886192448f646d72c137cbdb23f"
server_key_version: "encryptionkey@0"
I20260808 14:41:53.628036 3794 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 14:41:53.630601 3809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:53.631901 3794 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:41:53.632027 3794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "a93271875de240889bb59007953575ce"
format_stamp: "Formatted at 2026-08-08 14:41:53 on dist-test-slave-p3t5"
server_key: "d1dd45421e15d389754beff2fd012c23"
server_key_iv: "7fb0f886192448f646d72c137cbdb23f"
server_key_version: "encryptionkey@0"
I20260808 14:41:53.632124 3794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:41:53.663784 3794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:41:53.667232 3794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:41:53.667408 3794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:41:53.668184 3794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:41:53.669188 3794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:41:53.669250 3794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:53.669293 3794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:41:53.669317 3794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:41:53.678738 3794 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:36605
I20260808 14:41:53.678803 3927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:36605 every 8 connection(s)
I20260808 14:41:53.679769 3794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:41:53.688675 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 3794
I20260808 14:41:53.688824 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:41:53.689122 20033 external_mini_cluster.cc:1428] Setting key fbf76f68343ff9a35f61c5d8d72b0609
Aug 08 14:41:53 dist-test-slave-p3t5 krb5kdc[3411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200113, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:41:53.694666 3499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.681717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:58269 (local address 127.19.144.126:35049)
0808 14:41:53.681874 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:53.681878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:53.682791 (+ 913us) server_negotiation.cc:407] Connection header received
0808 14:41:53.684290 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:53.684294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:53.684361 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:53.684467 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:53.685815 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.686375 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.687632 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.687863 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.687989 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.691058 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:53.691062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:53.691065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:53.691096 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:53.692617 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:53.693163 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:53.693166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:53.693168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:53.693250 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:53.693565 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:53.693568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:53.693569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:53.693825 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:53.693907 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:53.694424 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 14:41:53.694539 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:41:53.695573 3930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.682136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:58269)
0808 14:41:53.682625 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:53.682671 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:53.684037 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:53.684675 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:53.684683 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:53.685076 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:41:53.685595 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:53.685668 (+ 73us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.686520 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.686524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:53.687483 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:53.687493 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.688029 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.688033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:53.688217 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.688961 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:41:53.688980 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:41:53.690888 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:41:53.692730 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:53.692736 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:53.692749 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:53.693064 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:53.693328 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:41:53.693330 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:41:53.693332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:41:53.693478 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:41:53.693936 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:41:53.693941 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:41:53.694201 (+ 260us) client_negotiation.cc:769] Sending connection context
0808 14:41:53.694358 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 14:41:53.694641 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":121,"threads_started":1}
I20260808 14:41:53.696830 3928 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35049
I20260808 14:41:53.697100 3928 heartbeater.cc:464] Registering TS with master...
I20260808 14:41:53.697665 3928 heartbeater.cc:511] Master 127.19.144.126:35049 requested a full tablet report, sending...
I20260808 14:41:53.698770 3458 ts_manager.cc:194] Registered new tserver with Master: a93271875de240889bb59007953575ce (127.19.144.67:36605)
I20260808 14:41:53.699357 3458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:58269
I20260808 14:41:53.703436 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:41:53.712937 3499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.705557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59894 (local address 127.19.144.126:35049)
0808 14:41:53.705736 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:53.705740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:53.705886 (+ 146us) server_negotiation.cc:407] Connection header received
0808 14:41:53.705989 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:53.705991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:53.706031 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:53.706100 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:41:53.706801 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.707309 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.708099 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.708301 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.708377 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.710086 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:41:53.710089 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:41:53.710091 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:41:53.710114 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:41:53.711482 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:53.711871 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:53.711874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:53.711875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:53.711954 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:41:53.712188 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:41:53.712190 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:41:53.712191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:41:53.712419 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:41:53.712538 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:53.712713 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 14:41:53.712811 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 14:41:53.716122 3458 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:41:53.719596 3458 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:41:53.734752 3944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.729231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55666 (local address 127.19.144.65:41059)
0808 14:41:53.729677 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:53.729686 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:53.729711 (+ 25us) server_negotiation.cc:407] Connection header received
0808 14:41:53.729895 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:53.729901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:53.730101 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:53.730246 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:53.730797 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.731523 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.733340 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.733995 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.734126 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.734295 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:53.734471 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 14:41:53.734579 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":113,"threads_started":1}
I20260808 14:41:53.734742 3939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.728481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:34323 (local address 127.0.0.1:40450)
0808 14:41:53.729148 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:53.729204 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:53.729342 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:53.730133 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:53.730139 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:53.730161 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:53.730315 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:53.730322 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.731742 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.731745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:53.733268 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:53.733279 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.734314 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.734318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:53.734405 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.734428 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 14:41:53.734483 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 14:41:53.734590 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":375,"thread_start_us":111,"threads_started":1}
I20260808 14:41:53.735188 3940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.728442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45536 (local address 127.19.144.67:36605)
0808 14:41:53.729919 (+ 1477us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:53.729930 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:53.729951 (+ 21us) server_negotiation.cc:407] Connection header received
0808 14:41:53.730029 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:53.730033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:53.730185 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:53.730311 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:53.730768 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.732113 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.733256 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.734139 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.734263 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.734424 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:53.734817 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 14:41:53.734910 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1376,"thread_start_us":150,"threads_started":1}
I20260808 14:41:53.735451 3941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.728298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:36605 (local address 127.0.0.1:45536)
0808 14:41:53.729654 (+ 1356us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:53.729669 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:53.729759 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:53.730458 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:53.730461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:53.730474 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:53.730638 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:53.730643 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.732284 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.732287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:53.733126 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:53.733136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.734486 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.734490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:53.734586 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.734606 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 14:41:53.734672 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 14:41:53.734742 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":38,"thread_start_us":98,"threads_started":1}
I20260808 14:41:53.735507 3943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.729150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40450 (local address 127.19.144.66:34323)
0808 14:41:53.729605 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:53.729640 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:53.729667 (+ 27us) server_negotiation.cc:407] Connection header received
0808 14:41:53.729754 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:53.729759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:53.729971 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:53.730140 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:53.730436 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.731545 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.733393 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.734181 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.734576 (+ 395us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.734721 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:53.734772 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 14:41:53.735153 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":117,"threads_started":1}
I20260808 14:41:53.735874 3942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.728708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:41059 (local address 127.0.0.1:55666)
0808 14:41:53.729606 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:53.729638 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:53.729735 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:53.730396 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:53.730399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:53.730417 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:53.730630 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:53.730639 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.731645 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.731651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:53.733184 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:53.733197 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.734126 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.734130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:53.734248 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.734287 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 14:41:53.734430 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 14:41:53.735735 (+ 1305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"mutex_wait_us":70,"thread_start_us":98,"threads_started":1}
I20260808 14:41:53.738615 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 7fae16a8133d43fb85040bef67143515 (DEFAULT_TABLE table=test-table [id=c174cfa543674c1e9133d478d8162e22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:41:53.738679 3723 tablet_service.cc:1467] Processing CreateTablet for tablet 7fae16a8133d43fb85040bef67143515 (DEFAULT_TABLE table=test-table [id=c174cfa543674c1e9133d478d8162e22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:41:53.738679 3586 tablet_service.cc:1467] Processing CreateTablet for tablet 7fae16a8133d43fb85040bef67143515 (DEFAULT_TABLE table=test-table [id=c174cfa543674c1e9133d478d8162e22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:41:53.739614 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fae16a8133d43fb85040bef67143515. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:53.739725 3586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fae16a8133d43fb85040bef67143515. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:53.739725 3723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fae16a8133d43fb85040bef67143515. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:41:53.746255 3945 tablet_bootstrap.cc:492] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce: Bootstrap starting.
I20260808 14:41:53.747999 3947 tablet_bootstrap.cc:492] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2: Bootstrap starting.
I20260808 14:41:53.748804 3945 tablet_bootstrap.cc:654] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:53.749970 3946 tablet_bootstrap.cc:492] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: Bootstrap starting.
I20260808 14:41:53.750470 3945 log.cc:824] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:53.750860 3947 tablet_bootstrap.cc:654] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:53.751690 3947 log.cc:824] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:53.752206 3946 tablet_bootstrap.cc:654] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: Neither blocks nor log segments found. Creating new log.
I20260808 14:41:53.752981 3945 tablet_bootstrap.cc:492] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce: No bootstrap required, opened a new log
I20260808 14:41:53.753026 3946 log.cc:824] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: Log is configured to *not* fsync() on all Append() calls
I20260808 14:41:53.753204 3945 ts_tablet_manager.cc:1397] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 14:41:53.753598 3947 tablet_bootstrap.cc:492] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2: No bootstrap required, opened a new log
I20260808 14:41:53.753825 3947 ts_tablet_manager.cc:1397] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:41:53.755210 3946 tablet_bootstrap.cc:492] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: No bootstrap required, opened a new log
I20260808 14:41:53.755410 3946 ts_tablet_manager.cc:1397] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:41:53.756771 3947 raft_consensus.cc:348] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:41:53.757051 3947 raft_consensus.cc:374] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:53.757108 3947 raft_consensus.cc:729] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1466ce81810245f099fcf8dfa47444d2, State: Initialized, Role: FOLLOWER
I20260808 14:41:53.757378 3945 raft_consensus.cc:348] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:41:53.757719 3945 raft_consensus.cc:374] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:53.757658 3947 consensus_queue.cc:260] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:41:53.757774 3945 raft_consensus.cc:729] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a93271875de240889bb59007953575ce, State: Initialized, Role: FOLLOWER
I20260808 14:41:53.758239 3945 consensus_queue.cc:260] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:41:53.758769 3947 ts_tablet_manager.cc:1428] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 14:41:53.759090 3789 heartbeater.cc:503] Master 127.19.144.126:35049 was elected leader, sending a full tablet report...
I20260808 14:41:53.759253 3945 ts_tablet_manager.cc:1428] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:41:53.759632 3928 heartbeater.cc:503] Master 127.19.144.126:35049 was elected leader, sending a full tablet report...
I20260808 14:41:53.760012 3946 raft_consensus.cc:348] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:41:53.760272 3946 raft_consensus.cc:374] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:41:53.760347 3946 raft_consensus.cc:729] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1e723868bc4464832909357429ad27, State: Initialized, Role: FOLLOWER
I20260808 14:41:53.760975 3946 consensus_queue.cc:260] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:41:53.761693 3654 heartbeater.cc:503] Master 127.19.144.126:35049 was elected leader, sending a full tablet report...
I20260808 14:41:53.762076 3946 ts_tablet_manager.cc:1428] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 14:41:53.898329 3951 raft_consensus.cc:482] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:41:53.898618 3951 raft_consensus.cc:504] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:41:53.899951 3951 leader_election.cc:288] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a93271875de240889bb59007953575ce (127.19.144.67:36605), 0e1e723868bc4464832909357429ad27 (127.19.144.65:41059)
I20260808 14:41:53.905159 3944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.900393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:37597 (local address 127.19.144.65:41059)
0808 14:41:53.900531 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:53.900536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:53.900554 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:41:53.900725 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:53.900729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:53.900791 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:53.900883 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:53.901265 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.902079 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.903757 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.904438 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.904570 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.904629 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:53.904930 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 14:41:53.905013 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:41:53.905159 3940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.900334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:41593 (local address 127.19.144.67:36605)
0808 14:41:53.900479 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:53.900483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:53.901157 (+ 674us) server_negotiation.cc:407] Connection header received
0808 14:41:53.901264 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:53.901267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:53.901319 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:53.901400 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:41:53.901752 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.902385 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.903760 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.904438 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.904570 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.904629 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:53.904937 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 14:41:53.905029 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 14:41:53.905175 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.900274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:41059 (local address 127.19.144.66:37597)
0808 14:41:53.900420 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:53.900439 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:53.900579 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:53.900960 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:53.900964 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:53.900989 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:53.901142 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:53.901149 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.902198 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.902200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:53.903574 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:53.903587 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.904570 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.904574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:53.904742 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.904809 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 14:41:53.904895 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 14:41:53.905027 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":65}
I20260808 14:41:53.905222 3954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.900279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:36605 (local address 127.19.144.66:41593)
0808 14:41:53.901022 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:53.901039 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:41:53.901158 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:41:53.901455 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:41:53.901458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:41:53.901474 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:41:53.901571 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:53.901576 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.902495 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.902498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:53.903645 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:41:53.903656 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.904570 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.904574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:41:53.904736 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.904810 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 14:41:53.904895 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 14:41:53.905012 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":484,"thread_start_us":91,"threads_started":1}
I20260808 14:41:53.905946 3881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fae16a8133d43fb85040bef67143515" candidate_uuid: "1466ce81810245f099fcf8dfa47444d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a93271875de240889bb59007953575ce" is_pre_election: true
I20260808 14:41:53.906311 3881 raft_consensus.cc:2454] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1466ce81810245f099fcf8dfa47444d2 in term 0.
I20260808 14:41:53.906366 3606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fae16a8133d43fb85040bef67143515" candidate_uuid: "1466ce81810245f099fcf8dfa47444d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1e723868bc4464832909357429ad27" is_pre_election: true
I20260808 14:41:53.906736 3606 raft_consensus.cc:2454] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1466ce81810245f099fcf8dfa47444d2 in term 0.
I20260808 14:41:53.906893 3675 leader_election.cc:302] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1466ce81810245f099fcf8dfa47444d2, a93271875de240889bb59007953575ce; no voters:
I20260808 14:41:53.907197 3951 raft_consensus.cc:2781] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:41:53.907282 3951 raft_consensus.cc:482] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:41:53.907332 3951 raft_consensus.cc:3037] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:53.908420 3951 raft_consensus.cc:504] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:41:53.908820 3951 leader_election.cc:288] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [CANDIDATE]: Term 1 election: Requested vote from peers a93271875de240889bb59007953575ce (127.19.144.67:36605), 0e1e723868bc4464832909357429ad27 (127.19.144.65:41059)
I20260808 14:41:53.909192 3881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fae16a8133d43fb85040bef67143515" candidate_uuid: "1466ce81810245f099fcf8dfa47444d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a93271875de240889bb59007953575ce"
I20260808 14:41:53.909329 3881 raft_consensus.cc:3037] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:53.909334 3606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fae16a8133d43fb85040bef67143515" candidate_uuid: "1466ce81810245f099fcf8dfa47444d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1e723868bc4464832909357429ad27"
I20260808 14:41:53.909483 3606 raft_consensus.cc:3037] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:41:53.910650 3881 raft_consensus.cc:2454] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1466ce81810245f099fcf8dfa47444d2 in term 1.
I20260808 14:41:53.910707 3606 raft_consensus.cc:2454] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1466ce81810245f099fcf8dfa47444d2 in term 1.
I20260808 14:41:53.911019 3675 leader_election.cc:302] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1466ce81810245f099fcf8dfa47444d2, a93271875de240889bb59007953575ce; no voters:
I20260808 14:41:53.911216 3951 raft_consensus.cc:2781] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:41:53.911478 3951 raft_consensus.cc:686] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [term 1 LEADER]: Becoming Leader. State: Replica: 1466ce81810245f099fcf8dfa47444d2, State: Running, Role: LEADER
I20260808 14:41:53.911849 3951 consensus_queue.cc:237] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:41:53.915241 3457 catalog_manager.cc:5495] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66). New cstate: current_term: 1 leader_uuid: "1466ce81810245f099fcf8dfa47444d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } health_report { overall_health: UNKNOWN } } }
W20260808 14:41:53.917902 3655 tablet.cc:2369] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:41:53.930861 3929 tablet.cc:2369] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:41:53.934228 3790 tablet.cc:2369] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:41:53.972922 3943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:53.969157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40464 (local address 127.19.144.66:34323)
0808 14:41:53.969295 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 14:41:53.969299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:41:53.969392 (+ 93us) server_negotiation.cc:407] Connection header received
0808 14:41:53.969650 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:41:53.969654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:41:53.969720 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:41:53.969815 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:41:53.970073 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.970658 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.971564 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:41:53.971772 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:41:53.971850 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:41:53.972135 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:41:53.972584 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:41:53.972679 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 14:41:53.972737 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 14:41:53.972787 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:41:53.979172 3881 raft_consensus.cc:1261] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce [term 1 FOLLOWER]: Refusing update from remote peer 1466ce81810245f099fcf8dfa47444d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:41:53.979172 3606 raft_consensus.cc:1261] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Refusing update from remote peer 1466ce81810245f099fcf8dfa47444d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:41:53.979933 3955 consensus_queue.cc:1035] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:41:53.980142 3951 consensus_queue.cc:1035] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:41:53.992255 3960 mvcc.cc:205] Tried to move back new op lower bound from 7316275666850529280 to 7316275666585784320. Current Snapshot: MvccSnapshot[applied={T|T < 7316275666850529280}]
I20260808 14:41:53.997537 3458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:41:53.997761 3458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:41:54.000335 3458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Sending DeleteTablet for 3 replicas of tablet 7fae16a8133d43fb85040bef67143515
I20260808 14:41:54.001197 3723 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fae16a8133d43fb85040bef67143515 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:41:53 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:40450
I20260808 14:41:54.001204 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fae16a8133d43fb85040bef67143515 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:41:53 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:45536
I20260808 14:41:54.001369 3586 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fae16a8133d43fb85040bef67143515 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:41:53 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:55666
I20260808 14:41:54.001674 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 3522
I20260808 14:41:54.004873 3967 tablet_replica.cc:330] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2: stopping tablet replica
I20260808 14:41:54.005264 3967 raft_consensus.cc:2229] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:41:54.005728 3967 raft_consensus.cc:2258] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:41:54.006150 3968 tablet_replica.cc:330] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce: stopping tablet replica
I20260808 14:41:54.006492 3968 raft_consensus.cc:2229] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:41:54.006876 3968 raft_consensus.cc:2258] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:41:54.008559 3967 ts_tablet_manager.cc:1905] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:41:54.008560 3968 ts_tablet_manager.cc:1905] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:41:54.011849 3967 ts_tablet_manager.cc:1918] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:41:54.011965 3967 log.cc:1196] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/7fae16a8133d43fb85040bef67143515
I20260808 14:41:54.012295 3967 ts_tablet_manager.cc:1939] T 7fae16a8133d43fb85040bef67143515 P 1466ce81810245f099fcf8dfa47444d2: Deleting consensus metadata
I20260808 14:41:54.012604 3968 ts_tablet_manager.cc:1918] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:41:54.012778 3968 log.cc:1196] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/7fae16a8133d43fb85040bef67143515
W20260808 14:41:54.012876 3444 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.144.65:41059 (error 108)
I20260808 14:41:54.013254 3968 ts_tablet_manager.cc:1939] T 7fae16a8133d43fb85040bef67143515 P a93271875de240889bb59007953575ce: Deleting consensus metadata
I20260808 14:41:54.013666 3443 catalog_manager.cc:4841] TS 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66:34323): tablet 7fae16a8133d43fb85040bef67143515 (table test-table [id=c174cfa543674c1e9133d478d8162e22]) successfully deleted
I20260808 14:41:54.014187 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 3659
I20260808 14:41:54.014513 3942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:41:54.014208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:41059 (local address 127.0.0.1:55682)
0808 14:41:54.014334 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 14:41:54.014412 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:41059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 14:41:54.016562 3442 catalog_manager.cc:4841] TS a93271875de240889bb59007953575ce (127.19.144.67:36605): tablet 7fae16a8133d43fb85040bef67143515 (table test-table [id=c174cfa543674c1e9133d478d8162e22]) successfully deleted
W20260808 14:41:54.017093 3444 catalog_manager.cc:4584] TS 0e1e723868bc4464832909357429ad27 (127.19.144.65:41059): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7fae16a8133d43fb85040bef67143515: Network error: Client connection negotiation failed: client connection to 127.19.144.65:41059: connect: Connection refused (error 111)
I20260808 14:41:54.023025 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 3794
I20260808 14:41:54.030493 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 3426
I20260808 14:42:14.038996 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:35049
--webserver_interface=127.19.144.126
--webserver_port=43281
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:35049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:42:14.160425 3971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:14.160810 3971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:14.160902 3971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:14.165175 3971 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 14:42:14.165288 3971 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 14:42:14.165318 3971 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 14:42:14.165339 3971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:42:14.165382 3971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:14.165446 3971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:14.165505 3971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:42:14.165546 3971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:42:14.171365 3971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:35049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:35049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=43281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:14.172588 3971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:14.173741 3971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:14.188802 3980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:14.188792 3977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:14.188979 3971 server_base.cc:1034] running on GCE node
W20260808 14:42:14.189194 3978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:14.190043 3971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:14.191313 3971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:14.192487 3971 hybrid_clock.cc:648] HybridClock initialized: now 1786200134192467 us; error 33 us; skew 500 ppm
Aug 08 14:42:14 dist-test-slave-p3t5 krb5kdc[3411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200134, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:14.195653 3971 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:42:14.196729 3971 webserver.cc:460] Webserver started at http://127.19.144.126:43281/ using document root <none> and password file <none>
I20260808 14:42:14.197427 3971 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:14.197548 3971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:14.201961 3971 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:42:14.204469 3987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:14.205986 3971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:42:14.206159 3971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "0d45fad648bc448893b80c7429c4ee52"
format_stamp: "Formatted at 2026-08-08 14:41:52 on dist-test-slave-p3t5"
server_key: "2f39c340e0d9bab58502875f05274937"
server_key_iv: "3a7d9de9ee07a7932c356b961296a232"
server_key_version: "encryptionkey@0"
I20260808 14:42:14.206831 3971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:14.222941 3971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:14.226902 3971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:14.227152 3971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:14.235498 3971 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:35049
I20260808 14:42:14.235834 4041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:35049 every 8 connection(s)
I20260808 14:42:14.237178 3971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:42:14.238355 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 3971
I20260808 14:42:14.238826 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:41059
--local_ip_for_outbound_sockets=127.19.144.65
--tserver_master_addrs=127.19.144.126:35049
--webserver_port=44069
--webserver_interface=127.19.144.65
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:42:14.242727 4042 sys_catalog.cc:263] Verifying existing consensus state
I20260808 14:42:14.244640 4042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Bootstrap starting.
I20260808 14:42:14.262669 4042 log.cc:824] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:14.271975 4042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:42:14.272444 4042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Bootstrap complete.
I20260808 14:42:14.276705 4042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } }
I20260808 14:42:14.277143 4042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d45fad648bc448893b80c7429c4ee52, State: Initialized, Role: FOLLOWER
I20260808 14:42:14.277763 4042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } }
I20260808 14:42:14.277897 4042 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:42:14.277994 4042 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:42:14.278113 4042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 1 FOLLOWER]: Advancing to term 2
I20260808 14:42:14.280202 4042 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } }
I20260808 14:42:14.280576 4042 leader_election.cc:302] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d45fad648bc448893b80c7429c4ee52; no voters:
I20260808 14:42:14.280932 4042 leader_election.cc:288] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 14:42:14.281092 4046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 14:42:14.281374 4046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 2 LEADER]: Becoming Leader. State: Replica: 0d45fad648bc448893b80c7429c4ee52, State: Running, Role: LEADER
I20260808 14:42:14.281744 4046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } }
I20260808 14:42:14.282330 4042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:42:14.283277 4047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0d45fad648bc448893b80c7429c4ee52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } } }
I20260808 14:42:14.283408 4047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [sys.catalog]: This master's current role is: LEADER
I20260808 14:42:14.283694 4048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d45fad648bc448893b80c7429c4ee52. Latest consensus state: current_term: 2 leader_uuid: "0d45fad648bc448893b80c7429c4ee52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } } }
I20260808 14:42:14.284322 4048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [sys.catalog]: This master's current role is: LEADER
I20260808 14:42:14.284431 4055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:42:14.288664 4055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:42:14.289244 4055 catalog_manager.cc:1259] Loaded cluster ID: 1b214b6b360142ca8c5cec785f370806
I20260808 14:42:14.289330 4055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:42:14.291275 4055 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:42:14.291916 4055 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Loaded TSK: 0
I20260808 14:42:14.307595 4055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Generated new TSK 1
I20260808 14:42:14.307928 4055 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 14:42:14.375451 4044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:14.375794 4044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:14.375881 4044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:14.380146 4044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:14.380260 4044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:14.380363 4044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:42:14.386627 4044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:41059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=44069
--tserver_master_addrs=127.19.144.126:35049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:14.387727 4044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:14.388787 4044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:14.399562 4070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:14.399564 4069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:14.399564 4072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:14.400110 4044 server_base.cc:1034] running on GCE node
I20260808 14:42:14.400563 4044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:14.401300 4044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:14.402494 4044 hybrid_clock.cc:648] HybridClock initialized: now 1786200134402457 us; error 56 us; skew 500 ppm
Aug 08 14:42:14 dist-test-slave-p3t5 krb5kdc[3411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200134, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:14.405601 4044 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:42:14.406694 4044 webserver.cc:460] Webserver started at http://127.19.144.65:44069/ using document root <none> and password file <none>
I20260808 14:42:14.407303 4044 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:14.407428 4044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:14.412019 4044 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 14:42:14.414548 4079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:14.415993 4044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:42:14.416147 4044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "0e1e723868bc4464832909357429ad27"
format_stamp: "Formatted at 2026-08-08 14:41:53 on dist-test-slave-p3t5"
server_key: "8c0fafcd93394c51ec44b13fcabd6597"
server_key_iv: "400ab80e369ded9b435fc00c96eaa1f5"
server_key_version: "encryptionkey@0"
I20260808 14:42:14.416810 4044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:14.441768 4044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:14.445132 4044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:14.445361 4044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:14.446112 4044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:14.447356 4086 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 14:42:14.451726 4044 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 14:42:14.451809 4044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260808 14:42:14.451912 4044 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 14:42:14.454121 4044 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 14:42:14.454200 4044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.002s
I20260808 14:42:14.454519 4086 tablet_bootstrap.cc:492] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: Bootstrap starting.
I20260808 14:42:14.464702 4044 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:41059
I20260808 14:42:14.464818 4196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:41059 every 8 connection(s)
I20260808 14:42:14.465853 4044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:42:14.467796 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 4044
I20260808 14:42:14.468304 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:34323
--local_ip_for_outbound_sockets=127.19.144.66
--tserver_master_addrs=127.19.144.126:35049
--webserver_port=45287
--webserver_interface=127.19.144.66
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:42:14.475970 4086 log.cc:824] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:42:14 dist-test-slave-p3t5 krb5kdc[3411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200134, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:42:14.485749 4086 tablet_bootstrap.cc:492] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:42:14.486229 4086 tablet_bootstrap.cc:492] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: Bootstrap complete.
I20260808 14:42:14.487593 4086 ts_tablet_manager.cc:1397] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: Time spent bootstrapping tablet: real 0.033s user 0.027s sys 0.004s
I20260808 14:42:14.488344 4199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:14.468228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:52093 (local address 127.19.144.126:35049)
0808 14:42:14.468988 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:14.469010 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:14.469487 (+ 477us) server_negotiation.cc:407] Connection header received
0808 14:42:14.471004 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:14.471022 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:14.471472 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:14.471923 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:14.473495 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.475113 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.477202 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.477983 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.478173 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:14.481791 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:14.481799 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:14.481816 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:14.481882 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:14.484276 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:14.485006 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:14.485014 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:14.485020 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:14.485220 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:14.485666 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:14.485670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:14.485672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:14.486220 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:14.486371 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:14.487071 (+ 700us) server_negotiation.cc:299] Negotiation successful
0808 14:42:14.487407 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":141,"threads_started":1}
I20260808 14:42:14.488369 4200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:14.468239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:52093)
0808 14:42:14.469295 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:14.469341 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:14.470427 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:14.471965 (+ 1538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:14.471978 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:14.472524 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:14.473241 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:14.473267 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.475312 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.475319 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:14.477012 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:14.477030 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.478159 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.478164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:14.478428 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:14.479215 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:42:14.479243 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:14.481483 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:14.484432 (+ 2949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:14.484439 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:14.484455 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:14.484837 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:14.485330 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:14.485333 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:14.485335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:14.485522 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:14.486382 (+ 860us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:14.486390 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:14.486695 (+ 305us) client_negotiation.cc:769] Sending connection context
0808 14:42:14.486926 (+ 231us) client_negotiation.cc:240] Negotiation successful
0808 14:42:14.487276 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"thread_start_us":166,"threads_started":1}
I20260808 14:42:14.490412 4086 raft_consensus.cc:348] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:14.490815 4086 raft_consensus.cc:729] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1e723868bc4464832909357429ad27, State: Initialized, Role: FOLLOWER
I20260808 14:42:14.491318 4086 consensus_queue.cc:260] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:14.491600 4197 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35049
I20260808 14:42:14.491847 4197 heartbeater.cc:464] Registering TS with master...
I20260808 14:42:14.492390 4086 ts_tablet_manager.cc:1428] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 14:42:14.492400 4197 heartbeater.cc:511] Master 127.19.144.126:35049 requested a full tablet report, sending...
I20260808 14:42:14.495222 4006 ts_manager.cc:194] Registered new tserver with Master: 0e1e723868bc4464832909357429ad27 (127.19.144.65:41059)
I20260808 14:42:14.499368 4006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:52093
I20260808 14:42:14.502373 4197 heartbeater.cc:503] Master 127.19.144.126:35049 was elected leader, sending a full tablet report...
Aug 08 14:42:14 dist-test-slave-p3t5 krb5kdc[3411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200134, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for kudu/127.19.144.65@KRBTEST.COM
I20260808 14:42:14.510185 4205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:14.498853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.19.144.65:41059)
0808 14:42:14.499625 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:14.499639 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:14.499663 (+ 24us) server_negotiation.cc:407] Connection header received
0808 14:42:14.499887 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:14.499891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:14.500054 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:14.500370 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:14.501131 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.502184 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.503341 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.503681 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.503828 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:14.506452 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:14.506461 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:14.506472 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:14.506494 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:14.508087 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:14.508619 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:14.508630 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:14.508631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:14.508712 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:14.509029 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:14.509032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:14.509034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:14.509295 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:14.509444 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:14.509857 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 14:42:14.510037 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":110,"threads_started":1}
I20260808 14:42:14.510181 4204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:14.498811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:41059 (local address 127.0.0.1:42584)
0808 14:42:14.499384 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:14.499434 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:14.499702 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:14.500363 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:14.500369 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:14.500848 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:14.500984 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:14.500993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.502341 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.502345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:14.503212 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:14.503221 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.503845 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.503849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:14.503967 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:14.504630 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:42:14.504661 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:14.506294 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:14.508213 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:14.508224 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:14.508229 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:14.508491 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:14.508823 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:14.508826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:14.508828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:14.508925 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:14.509419 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:14.509426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:14.509563 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 14:42:14.509752 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 14:42:14.509980 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":118,"threads_started":1}
I20260808 14:42:14.511312 4129 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fae16a8133d43fb85040bef67143515 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:41:53 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:42584
I20260808 14:42:14.512462 4206 tablet_replica.cc:330] stopping tablet replica
I20260808 14:42:14.512800 4206 raft_consensus.cc:2229] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:42:14.513087 4206 raft_consensus.cc:2258] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:42:14.514230 4206 ts_tablet_manager.cc:1905] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:42:14.518204 4206 ts_tablet_manager.cc:1918] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:42:14.518349 4206 log.cc:1196] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal/wals/7fae16a8133d43fb85040bef67143515
I20260808 14:42:14.518662 4206 ts_tablet_manager.cc:1939] T 7fae16a8133d43fb85040bef67143515 P 0e1e723868bc4464832909357429ad27: Deleting consensus metadata
I20260808 14:42:14.520625 3990 catalog_manager.cc:4841] TS 0e1e723868bc4464832909357429ad27 (127.19.144.65:41059): tablet 7fae16a8133d43fb85040bef67143515 (table test-table [id=c174cfa543674c1e9133d478d8162e22]) successfully deleted
W20260808 14:42:14.603216 4201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:14.603534 4201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:14.603616 4201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:14.608039 4201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:14.608119 4201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:14.608182 4201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:42:14.614073 4201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:34323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=45287
--tserver_master_addrs=127.19.144.126:35049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:14.615372 4201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:14.616629 4201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:14.627805 4212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:14.627825 4211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:14.627964 4201 server_base.cc:1034] running on GCE node
W20260808 14:42:14.628201 4214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:14.628675 4201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:14.629415 4201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:14.630621 4201 hybrid_clock.cc:648] HybridClock initialized: now 1786200134630574 us; error 66 us; skew 500 ppm
Aug 08 14:42:14 dist-test-slave-p3t5 krb5kdc[3411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200134, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:14.633796 4201 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:42:14.634766 4201 webserver.cc:460] Webserver started at http://127.19.144.66:45287/ using document root <none> and password file <none>
I20260808 14:42:14.635344 4201 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:14.635424 4201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:14.639724 4201 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:42:14.642037 4221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:14.643412 4201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:42:14.643517 4201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "1466ce81810245f099fcf8dfa47444d2"
format_stamp: "Formatted at 2026-08-08 14:41:53 on dist-test-slave-p3t5"
server_key: "c188e8471ab5d02c4986db89502f2be6"
server_key_iv: "c0806f8b9eac0b4c34716cef43e51b39"
server_key_version: "encryptionkey@0"
I20260808 14:42:14.644146 4201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:14.663009 4201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:14.666405 4201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:14.666628 4201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:14.667339 4201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:14.668262 4201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:42:14.668340 4201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:14.668412 4201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:42:14.668458 4201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:14.678937 4201 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:34323
I20260808 14:42:14.679013 4337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:34323 every 8 connection(s)
I20260808 14:42:14.680101 4201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:42:14.688071 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 4201
I20260808 14:42:14.688611 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:36605
--local_ip_for_outbound_sockets=127.19.144.67
--tserver_master_addrs=127.19.144.126:35049
--webserver_port=33425
--webserver_interface=127.19.144.67
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 14:42:14 dist-test-slave-p3t5 krb5kdc[3411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200134, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:42:14.696487 4199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:14.682203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:51633 (local address 127.19.144.126:35049)
0808 14:42:14.682360 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:14.682364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:14.683420 (+ 1056us) server_negotiation.cc:407] Connection header received
0808 14:42:14.685027 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:14.685031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:14.685099 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:14.685211 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:14.686591 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.687167 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.688759 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.689084 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.689184 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:14.692201 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:14.692207 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:14.692213 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:14.692250 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:14.694112 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:14.694635 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:14.694641 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:14.694646 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:14.694753 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:14.695112 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:14.695117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:14.695122 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:14.695409 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:14.695532 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:14.696063 (+ 531us) server_negotiation.cc:299] Negotiation successful
0808 14:42:14.696233 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:42:14.697307 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:14.682642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:51633)
0808 14:42:14.683252 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:14.683303 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:14.684770 (+ 1467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:14.685416 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:14.685423 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:14.685898 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:14.686424 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:14.686439 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.687312 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.687315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:14.688584 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:14.688596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.689273 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.689279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:14.689537 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:14.690249 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:42:14.690275 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:14.692001 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:14.694234 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:14.694240 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:14.694254 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:14.694525 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:14.694861 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:14.694864 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:14.694865 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:14.695009 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:14.695530 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:14.695536 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:14.695800 (+ 264us) client_negotiation.cc:769] Sending connection context
0808 14:42:14.695971 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 14:42:14.696247 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":171,"threads_started":1}
I20260808 14:42:14.698710 4338 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35049
I20260808 14:42:14.699013 4338 heartbeater.cc:464] Registering TS with master...
I20260808 14:42:14.699545 4338 heartbeater.cc:511] Master 127.19.144.126:35049 requested a full tablet report, sending...
I20260808 14:42:14.700730 4006 ts_manager.cc:194] Registered new tserver with Master: 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66:34323)
I20260808 14:42:14.701689 4006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:51633
W20260808 14:42:14.812111 4341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:14.812426 4341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:14.812474 4341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:14.816696 4341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:14.816776 4341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:14.816839 4341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:42:14.822634 4341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:36605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=33425
--tserver_master_addrs=127.19.144.126:35049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:14.823760 4341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:14.824793 4341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:14.835744 4347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:14.835745 4349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:14.835744 4346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:14.836031 4341 server_base.cc:1034] running on GCE node
I20260808 14:42:14.836591 4341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:14.837390 4341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:14.838609 4341 hybrid_clock.cc:648] HybridClock initialized: now 1786200134838573 us; error 47 us; skew 500 ppm
Aug 08 14:42:14 dist-test-slave-p3t5 krb5kdc[3411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200134, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:14.841725 4341 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:42:14.842717 4341 webserver.cc:460] Webserver started at http://127.19.144.67:33425/ using document root <none> and password file <none>
I20260808 14:42:14.843353 4341 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:14.843479 4341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:14.847738 4341 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:42:14.850015 4356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:14.851325 4341 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:42:14.851485 4341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "a93271875de240889bb59007953575ce"
format_stamp: "Formatted at 2026-08-08 14:41:53 on dist-test-slave-p3t5"
server_key: "d1dd45421e15d389754beff2fd012c23"
server_key_iv: "7fb0f886192448f646d72c137cbdb23f"
server_key_version: "encryptionkey@0"
I20260808 14:42:14.852077 4341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:14.897966 4341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:14.901791 4341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:14.902029 4341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:14.902695 4341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:14.903607 4341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:42:14.903684 4341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:14.903755 4341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:42:14.903807 4341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:14.913301 4341 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:36605
I20260808 14:42:14.913326 4471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:36605 every 8 connection(s)
I20260808 14:42:14.914441 4341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:42:14.918293 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 4341
Aug 08 14:42:14 dist-test-slave-p3t5 krb5kdc[3411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200134, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:42:14.928365 4476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:14.919053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42172 (local address 127.19.144.126:35049)
0808 14:42:14.919399 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:14.919406 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:14.919469 (+ 63us) server_negotiation.cc:407] Connection header received
0808 14:42:14.919662 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:14.919666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:14.919703 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:14.919781 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:14.920749 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.921523 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.922716 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.922950 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.923029 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:14.924751 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:14.924755 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:14.924757 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:14.924784 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:14.926352 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:14.926830 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:14.926833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:14.926835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:14.926940 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:14.927263 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:14.927270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:14.927274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:14.927571 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:14.927798 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:14.927958 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 14:42:14.928142 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":80,"threads_started":1}
I20260808 14:42:14.930680 4199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:14.916349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:53919 (local address 127.19.144.126:35049)
0808 14:42:14.916523 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:14.916528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:14.917474 (+ 946us) server_negotiation.cc:407] Connection header received
0808 14:42:14.919312 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:14.919315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:14.919371 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:14.919472 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:14.921140 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.921814 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.923343 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.923545 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.923655 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:14.926587 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:14.926590 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:14.926591 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:14.926613 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:14.928165 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:14.928833 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:14.928836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:14.928837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:14.928984 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:14.929380 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:14.929383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:14.929384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:14.929613 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:14.929728 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:14.930391 (+ 663us) server_negotiation.cc:299] Negotiation successful
0808 14:42:14.930526 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 14:42:14.931622 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:14.916823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:53919)
0808 14:42:14.917311 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:14.917357 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:14.919053 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:14.919727 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:14.919735 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:14.920339 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:14.921002 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:14.921018 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.921955 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.921959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:14.923218 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:14.923230 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.923716 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.923723 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:14.923907 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:14.924502 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:42:14.924519 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:14.926393 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:14.928294 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:14.928301 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:14.928317 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:14.928669 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:14.929087 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:14.929090 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:14.929092 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:14.929256 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:14.929749 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:14.929755 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:14.930084 (+ 329us) client_negotiation.cc:769] Sending connection context
0808 14:42:14.930304 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 14:42:14.930686 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":118,"threads_started":1}
I20260808 14:42:14.932696 4472 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35049
I20260808 14:42:14.932969 4472 heartbeater.cc:464] Registering TS with master...
I20260808 14:42:14.933511 4472 heartbeater.cc:511] Master 127.19.144.126:35049 requested a full tablet report, sending...
I20260808 14:42:14.934609 4006 ts_manager.cc:194] Registered new tserver with Master: a93271875de240889bb59007953575ce (127.19.144.67:36605)
I20260808 14:42:14.935190 4006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:53919
I20260808 14:42:14.942411 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:42:14.946346 4199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:14.943059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42174 (local address 127.19.144.126:35049)
0808 14:42:14.943194 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:14.943198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:14.943355 (+ 157us) server_negotiation.cc:407] Connection header received
0808 14:42:14.943512 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:14.943514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:14.943620 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:14.943707 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:42:14.943901 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.944392 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.945211 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.945434 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.945511 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:14.945862 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:42:14.946017 (+ 155us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 14:42:14.946239 (+ 222us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42174: token expired
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 14:42:14.946419 4199 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42174: token expired
W20260808 14:42:14.946836 3932 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.19.144.126:35049)
I20260808 14:42:14.954892 4199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:14.947253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42188 (local address 127.19.144.126:35049)
0808 14:42:14.947386 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:14.947393 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:14.947407 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:42:14.947563 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:14.947566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:14.947618 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:14.947701 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:14.948360 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.948959 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.949872 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.950107 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.950217 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:14.951685 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:14.951688 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:14.951691 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:14.951719 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:14.953107 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:14.953568 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:14.953571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:14.953573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:14.953691 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:14.953998 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:14.954002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:14.954004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:14.954197 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:14.954338 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:14.954606 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 14:42:14.954730 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:42:14.957697 4006 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:42:14.961045 4006 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:42:14.969619 4129 tablet_service.cc:1467] Processing CreateTablet for tablet 57764198e4ec482fb293c62c80036d1a (DEFAULT_TABLE table=test-table [id=94f07a713c33426dac321c0c1ae693ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:42:14.970381 4129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57764198e4ec482fb293c62c80036d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:14.973243 4086 tablet_bootstrap.cc:492] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27: Bootstrap starting.
I20260808 14:42:14.974208 4204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:14.968332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:36605 (local address 127.0.0.1:41988)
0808 14:42:14.968475 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:14.968494 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:14.968578 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:14.969747 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:14.969750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:14.969770 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:14.969895 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:14.969903 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.971239 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.971242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:14.972609 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:14.972621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.973706 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.973710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:14.973820 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:14.973882 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 14:42:14.973952 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 14:42:14.974048 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":8}
I20260808 14:42:14.974300 4086 tablet_bootstrap.cc:654] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:14.974407 4482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:14.968776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:34323 (local address 127.0.0.1:46118)
0808 14:42:14.969093 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:14.969108 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:14.969202 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:14.970101 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:14.970103 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:14.970124 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:14.970206 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:14.970212 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.971498 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.971502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:14.972632 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:14.972639 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.973959 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.973966 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:14.974084 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:14.974110 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 14:42:14.974162 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 14:42:14.974240 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":27,"thread_start_us":88,"threads_started":1}
I20260808 14:42:14.974977 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:14.968551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41988 (local address 127.19.144.67:36605)
0808 14:42:14.969142 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:14.969156 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:14.969182 (+ 26us) server_negotiation.cc:407] Connection header received
0808 14:42:14.969271 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:14.969277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:14.969534 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:14.969775 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:14.970030 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.971106 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.972778 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.973536 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.974407 (+ 871us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:14.974596 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:14.974666 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 14:42:14.974800 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":175,"threads_started":1}
I20260808 14:42:14.975378 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:14.969200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46118 (local address 127.19.144.66:34323)
0808 14:42:14.969601 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:14.969630 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:14.969657 (+ 27us) server_negotiation.cc:407] Connection header received
0808 14:42:14.969739 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:14.969745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:14.969958 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:14.970135 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:14.970322 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.971375 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.972776 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:14.973792 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:14.974948 (+ 1156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:14.975112 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:14.975168 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 14:42:14.975265 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":179,"threads_started":1}
I20260808 14:42:14.976241 4086 tablet_bootstrap.cc:492] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27: No bootstrap required, opened a new log
I20260808 14:42:14.976377 4086 ts_tablet_manager.cc:1397] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 14:42:14.976897 4086 raft_consensus.cc:348] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:14.977036 4086 raft_consensus.cc:374] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:14.977290 4086 raft_consensus.cc:729] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1e723868bc4464832909357429ad27, State: Initialized, Role: FOLLOWER
I20260808 14:42:14.977471 4086 consensus_queue.cc:260] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:14.977773 4086 ts_tablet_manager.cc:1428] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 14:42:14.978475 4260 tablet_service.cc:1467] Processing CreateTablet for tablet 57764198e4ec482fb293c62c80036d1a (DEFAULT_TABLE table=test-table [id=94f07a713c33426dac321c0c1ae693ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:42:14.978366 4404 tablet_service.cc:1467] Processing CreateTablet for tablet 57764198e4ec482fb293c62c80036d1a (DEFAULT_TABLE table=test-table [id=94f07a713c33426dac321c0c1ae693ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:42:14.979497 4260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57764198e4ec482fb293c62c80036d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:14.979573 4404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57764198e4ec482fb293c62c80036d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:14.987291 4486 tablet_bootstrap.cc:492] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce: Bootstrap starting.
I20260808 14:42:14.987905 4487 tablet_bootstrap.cc:492] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2: Bootstrap starting.
I20260808 14:42:14.990202 4486 tablet_bootstrap.cc:654] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:14.990597 4487 tablet_bootstrap.cc:654] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:14.991374 4487 log.cc:824] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:14.991386 4486 log.cc:824] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:14.993458 4487 tablet_bootstrap.cc:492] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2: No bootstrap required, opened a new log
I20260808 14:42:14.993479 4486 tablet_bootstrap.cc:492] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce: No bootstrap required, opened a new log
I20260808 14:42:14.993739 4487 ts_tablet_manager.cc:1397] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:42:14.993742 4486 ts_tablet_manager.cc:1397] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 14:42:14.997174 4486 raft_consensus.cc:348] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:14.997166 4487 raft_consensus.cc:348] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:14.997437 4486 raft_consensus.cc:374] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:14.997464 4487 raft_consensus.cc:374] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:14.997500 4486 raft_consensus.cc:729] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a93271875de240889bb59007953575ce, State: Initialized, Role: FOLLOWER
I20260808 14:42:14.997547 4487 raft_consensus.cc:729] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1466ce81810245f099fcf8dfa47444d2, State: Initialized, Role: FOLLOWER
I20260808 14:42:14.998051 4486 consensus_queue.cc:260] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:14.998050 4487 consensus_queue.cc:260] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:14.998775 4338 heartbeater.cc:503] Master 127.19.144.126:35049 was elected leader, sending a full tablet report...
I20260808 14:42:14.998792 4472 heartbeater.cc:503] Master 127.19.144.126:35049 was elected leader, sending a full tablet report...
I20260808 14:42:14.999169 4487 ts_tablet_manager.cc:1428] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 14:42:14.999169 4486 ts_tablet_manager.cc:1428] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 14:42:15.022360 4490 raft_consensus.cc:482] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:42:15.022662 4490 raft_consensus.cc:504] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:15.023977 4490 leader_election.cc:288] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a93271875de240889bb59007953575ce (127.19.144.67:36605), 0e1e723868bc4464832909357429ad27 (127.19.144.65:41059)
I20260808 14:42:15.024484 4491 raft_consensus.cc:482] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:42:15.024783 4491 raft_consensus.cc:504] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:15.027210 4491 leader_election.cc:288] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66:34323), 0e1e723868bc4464832909357429ad27 (127.19.144.65:41059)
I20260808 14:42:15.029006 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:15.024185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:36605 (local address 127.19.144.66:60297)
0808 14:42:15.024378 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:15.024406 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:15.024503 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:15.024902 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:15.024905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:15.024924 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:15.025047 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:15.025053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.026080 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.026083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:15.027614 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:15.027624 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.028590 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.028594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:15.028703 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:15.028725 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 14:42:15.028792 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 14:42:15.028857 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":324}
I20260808 14:42:15.029452 4492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:15.024217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:41059 (local address 127.19.144.66:54129)
0808 14:42:15.024694 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:15.024710 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:15.024815 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:15.025189 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:15.025191 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:15.025206 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:15.025308 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:15.025313 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.026119 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.026126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:15.027501 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:15.027515 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.028669 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.028672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:15.028779 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:15.028802 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 14:42:15.028874 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 14:42:15.028939 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":152,"threads_started":1}
I20260808 14:42:15.029513 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:15.024317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:60297 (local address 127.19.144.67:36605)
0808 14:42:15.024426 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:15.024433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:15.024449 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:42:15.024638 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:15.024641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:15.024700 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:15.024775 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:15.025177 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.025948 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.027747 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.028354 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.028482 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:15.028533 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:15.029292 (+ 759us) server_negotiation.cc:299] Negotiation successful
0808 14:42:15.029412 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":36}
I20260808 14:42:15.029811 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:15.024582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:54129 (local address 127.19.144.65:41059)
0808 14:42:15.024903 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:15.024907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:15.024920 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:42:15.024969 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:15.024972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:15.025038 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:15.025145 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:15.025427 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.025984 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.027654 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.028318 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.028455 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:15.028602 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:15.029058 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 14:42:15.029357 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":228,"thread_start_us":111,"threads_started":1}
I20260808 14:42:15.030196 4425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57764198e4ec482fb293c62c80036d1a" candidate_uuid: "1466ce81810245f099fcf8dfa47444d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a93271875de240889bb59007953575ce" is_pre_election: true
I20260808 14:42:15.030472 4425 raft_consensus.cc:2454] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1466ce81810245f099fcf8dfa47444d2 in term 0.
I20260808 14:42:15.030532 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:15.026352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:42671 (local address 127.19.144.66:34323)
0808 14:42:15.026459 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:15.026464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:15.026476 (+ 12us) server_negotiation.cc:407] Connection header received
0808 14:42:15.026612 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:15.026615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:15.026655 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:15.026735 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:15.027083 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.027747 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.029116 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.029733 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.029836 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:15.029882 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:15.030349 (+ 467us) server_negotiation.cc:299] Negotiation successful
0808 14:42:15.030422 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 14:42:15.030722 4150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57764198e4ec482fb293c62c80036d1a" candidate_uuid: "1466ce81810245f099fcf8dfa47444d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1e723868bc4464832909357429ad27" is_pre_election: true
I20260808 14:42:15.031020 4222 leader_election.cc:302] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1466ce81810245f099fcf8dfa47444d2, a93271875de240889bb59007953575ce; no voters:
I20260808 14:42:15.031100 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:15.026282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:34323 (local address 127.19.144.67:42671)
0808 14:42:15.026404 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:15.026419 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:15.026502 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:15.026831 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:15.026834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:15.026853 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:15.026962 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:15.026967 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.027855 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.027859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:15.028977 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:15.028987 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.030125 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.030128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:15.030232 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:15.030255 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 14:42:15.030332 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 14:42:15.030405 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":1}
I20260808 14:42:15.031042 4150 raft_consensus.cc:2454] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1466ce81810245f099fcf8dfa47444d2 in term 0.
I20260808 14:42:15.031330 4490 raft_consensus.cc:2781] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:42:15.031437 4490 raft_consensus.cc:482] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:42:15.031500 4490 raft_consensus.cc:3037] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:15.031862 4289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57764198e4ec482fb293c62c80036d1a" candidate_uuid: "a93271875de240889bb59007953575ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1466ce81810245f099fcf8dfa47444d2" is_pre_election: true
I20260808 14:42:15.032433 4494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:15.026684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:41059 (local address 127.19.144.67:33657)
0808 14:42:15.027670 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:15.027684 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:15.027800 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:15.028131 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:15.028134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:15.028150 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:15.028267 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:15.028273 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.029203 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.029205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:15.030639 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:15.030649 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.032069 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.032072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:15.032174 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:15.032204 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 14:42:15.032268 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 14:42:15.032319 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":922,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260808 14:42:15.032552 4495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:15.027040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:33657 (local address 127.19.144.65:41059)
0808 14:42:15.027483 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:15.027487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:15.027743 (+ 256us) server_negotiation.cc:407] Connection header received
0808 14:42:15.027919 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:15.027922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:15.027965 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:15.028017 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:15.028392 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.029061 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.030754 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.031909 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.032032 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:15.032077 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:15.032312 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 14:42:15.032390 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":90,"threads_started":1}
I20260808 14:42:15.032920 4490 raft_consensus.cc:504] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:15.032963 4150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57764198e4ec482fb293c62c80036d1a" candidate_uuid: "a93271875de240889bb59007953575ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1e723868bc4464832909357429ad27" is_pre_election: true
I20260808 14:42:15.033125 4150 raft_consensus.cc:2454] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a93271875de240889bb59007953575ce in term 0.
I20260808 14:42:15.033198 4289 raft_consensus.cc:2379] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a93271875de240889bb59007953575ce in current term 1: Already voted for candidate 1466ce81810245f099fcf8dfa47444d2 in this term.
I20260808 14:42:15.033335 4490 leader_election.cc:288] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [CANDIDATE]: Term 1 election: Requested vote from peers a93271875de240889bb59007953575ce (127.19.144.67:36605), 0e1e723868bc4464832909357429ad27 (127.19.144.65:41059)
I20260808 14:42:15.033499 4359 leader_election.cc:302] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e1e723868bc4464832909357429ad27, a93271875de240889bb59007953575ce; no voters:
I20260808 14:42:15.033833 4425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57764198e4ec482fb293c62c80036d1a" candidate_uuid: "1466ce81810245f099fcf8dfa47444d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a93271875de240889bb59007953575ce"
I20260808 14:42:15.033844 4150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57764198e4ec482fb293c62c80036d1a" candidate_uuid: "1466ce81810245f099fcf8dfa47444d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1e723868bc4464832909357429ad27"
I20260808 14:42:15.033952 4425 raft_consensus.cc:3037] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:15.034004 4150 raft_consensus.cc:3037] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:15.035223 4425 raft_consensus.cc:2454] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1466ce81810245f099fcf8dfa47444d2 in term 1.
I20260808 14:42:15.035339 4150 raft_consensus.cc:2454] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1466ce81810245f099fcf8dfa47444d2 in term 1.
I20260808 14:42:15.035538 4491 raft_consensus.cc:2741] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260808 14:42:15.035614 4222 leader_election.cc:302] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1466ce81810245f099fcf8dfa47444d2, a93271875de240889bb59007953575ce; no voters:
I20260808 14:42:15.035781 4490 raft_consensus.cc:2781] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:42:15.036067 4490 raft_consensus.cc:686] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [term 1 LEADER]: Becoming Leader. State: Replica: 1466ce81810245f099fcf8dfa47444d2, State: Running, Role: LEADER
I20260808 14:42:15.036413 4490 consensus_queue.cc:237] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:15.039791 4006 catalog_manager.cc:5495] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66). New cstate: current_term: 1 leader_uuid: "1466ce81810245f099fcf8dfa47444d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } health_report { overall_health: UNKNOWN } } }
I20260808 14:42:15.059882 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:15.056095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46130 (local address 127.19.144.66:34323)
0808 14:42:15.056235 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:15.056239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:15.056253 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:42:15.056333 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:15.056336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:15.056392 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:15.056498 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:42:15.056829 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.057371 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.058387 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:15.058611 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:15.058703 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:15.059054 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:42:15.059505 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:42:15.059602 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:15.059685 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 14:42:15.059736 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:42:15.066695 4425 raft_consensus.cc:1261] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce [term 1 FOLLOWER]: Refusing update from remote peer 1466ce81810245f099fcf8dfa47444d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:42:15.066694 4150 raft_consensus.cc:1261] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Refusing update from remote peer 1466ce81810245f099fcf8dfa47444d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:42:15.067523 4490 consensus_queue.cc:1035] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:42:15.067793 4496 consensus_queue.cc:1035] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:42:15.075389 4498 mvcc.cc:205] Tried to move back new op lower bound from 7316275753224249344 to 7316275753112117248. Current Snapshot: MvccSnapshot[applied={T|T < 7316275753224249344}]
I20260808 14:42:15.075832 4499 mvcc.cc:205] Tried to move back new op lower bound from 7316275753224249344 to 7316275753112117248. Current Snapshot: MvccSnapshot[applied={T|T < 7316275753224249344}]
I20260808 14:42:15.086246 4004 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:42:15.086483 4004 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:42:15.089155 4004 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Sending DeleteTablet for 3 replicas of tablet 57764198e4ec482fb293c62c80036d1a
I20260808 14:42:15.089958 4129 tablet_service.cc:1514] Processing DeleteTablet for tablet 57764198e4ec482fb293c62c80036d1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:42:15 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:42584
I20260808 14:42:15.089979 4404 tablet_service.cc:1514] Processing DeleteTablet for tablet 57764198e4ec482fb293c62c80036d1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:42:15 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:41988
I20260808 14:42:15.090415 4260 tablet_service.cc:1514] Processing DeleteTablet for tablet 57764198e4ec482fb293c62c80036d1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:42:15 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:46118
I20260808 14:42:15.090746 4509 tablet_replica.cc:330] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce: stopping tablet replica
I20260808 14:42:15.090972 4510 tablet_replica.cc:330] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2: stopping tablet replica
I20260808 14:42:15.091125 4509 raft_consensus.cc:2229] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:42:15.091382 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 4044
I20260808 14:42:15.091514 4509 raft_consensus.cc:2258] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:42:15.091409 4510 raft_consensus.cc:2229] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:42:15.093152 4510 raft_consensus.cc:2258] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:42:15.094269 4509 ts_tablet_manager.cc:1905] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:42:15.094873 4510 ts_tablet_manager.cc:1905] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:42:15.097337 4509 ts_tablet_manager.cc:1918] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:42:15.097455 4509 log.cc:1196] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/57764198e4ec482fb293c62c80036d1a
I20260808 14:42:15.097855 4509 ts_tablet_manager.cc:1939] T 57764198e4ec482fb293c62c80036d1a P a93271875de240889bb59007953575ce: Deleting consensus metadata
I20260808 14:42:15.098800 4510 ts_tablet_manager.cc:1918] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:42:15.098978 4510 log.cc:1196] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/57764198e4ec482fb293c62c80036d1a
I20260808 14:42:15.099339 3988 catalog_manager.cc:4841] TS a93271875de240889bb59007953575ce (127.19.144.67:36605): tablet 57764198e4ec482fb293c62c80036d1a (table test-table [id=94f07a713c33426dac321c0c1ae693ce]) successfully deleted
I20260808 14:42:15.099406 4510 ts_tablet_manager.cc:1939] T 57764198e4ec482fb293c62c80036d1a P 1466ce81810245f099fcf8dfa47444d2: Deleting consensus metadata
I20260808 14:42:15.100881 3989 catalog_manager.cc:4841] TS 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66:34323): tablet 57764198e4ec482fb293c62c80036d1a (table test-table [id=94f07a713c33426dac321c0c1ae693ce]) successfully deleted
W20260808 14:42:15.102684 3990 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.144.65:41059 (error 108)
I20260808 14:42:15.103744 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 4201
I20260808 14:42:15.103919 4482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:15.103535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:41059 (local address 127.0.0.1:42588)
0808 14:42:15.103762 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:15.103833 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:41059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
W20260808 14:42:15.104969 3990 catalog_manager.cc:4584] TS 0e1e723868bc4464832909357429ad27 (127.19.144.65:41059): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 57764198e4ec482fb293c62c80036d1a: Network error: Client connection negotiation failed: client connection to 127.19.144.65:41059: connect: Connection refused (error 111)
I20260808 14:42:15.112533 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 4341
I20260808 14:42:15.120270 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 3971
I20260808 14:42:35.128818 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:35049
--webserver_interface=127.19.144.126
--webserver_port=43281
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:35049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:42:35.251389 4512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:35.251689 4512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:35.251732 4512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:35.256048 4512 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 14:42:35.256115 4512 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 14:42:35.256131 4512 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 14:42:35.256145 4512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:42:35.256165 4512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:35.256183 4512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:35.256228 4512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:42:35.256263 4512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:42:35.261860 4512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:35049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:35049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=43281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:35.263041 4512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:35.264122 4512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:35.273595 4517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:35.273700 4512 server_base.cc:1034] running on GCE node
W20260808 14:42:35.273762 4518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:35.273643 4520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:35.274550 4512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:35.275870 4512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:35.277079 4512 hybrid_clock.cc:648] HybridClock initialized: now 1786200155277041 us; error 49 us; skew 500 ppm
Aug 08 14:42:35 dist-test-slave-p3t5 krb5kdc[3411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200155, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:35.280349 4512 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:42:35.281375 4512 webserver.cc:460] Webserver started at http://127.19.144.126:43281/ using document root <none> and password file <none>
I20260808 14:42:35.282052 4512 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:35.282157 4512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:35.286417 4512 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:42:35.288882 4527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:35.290333 4512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:42:35.290510 4512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "0d45fad648bc448893b80c7429c4ee52"
format_stamp: "Formatted at 2026-08-08 14:41:52 on dist-test-slave-p3t5"
server_key: "2f39c340e0d9bab58502875f05274937"
server_key_iv: "3a7d9de9ee07a7932c356b961296a232"
server_key_version: "encryptionkey@0"
I20260808 14:42:35.291116 4512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:35.313019 4512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:35.317816 4512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:35.318040 4512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:35.325500 4512 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:35049
I20260808 14:42:35.325829 4580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:35049 every 8 connection(s)
I20260808 14:42:35.326776 4512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:42:35.327538 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 4512
I20260808 14:42:35.328015 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:41059
--local_ip_for_outbound_sockets=127.19.144.65
--tserver_master_addrs=127.19.144.126:35049
--webserver_port=44069
--webserver_interface=127.19.144.65
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:42:35.331575 4581 sys_catalog.cc:263] Verifying existing consensus state
I20260808 14:42:35.333555 4581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Bootstrap starting.
I20260808 14:42:35.352083 4581 log.cc:824] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:35.363811 4581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:42:35.364281 4581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Bootstrap complete.
I20260808 14:42:35.368536 4581 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } }
I20260808 14:42:35.368954 4581 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d45fad648bc448893b80c7429c4ee52, State: Initialized, Role: FOLLOWER
I20260808 14:42:35.369470 4581 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } }
I20260808 14:42:35.369604 4581 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:42:35.369728 4581 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:42:35.369864 4581 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 2 FOLLOWER]: Advancing to term 3
I20260808 14:42:35.371923 4581 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } }
I20260808 14:42:35.372292 4581 leader_election.cc:302] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d45fad648bc448893b80c7429c4ee52; no voters:
I20260808 14:42:35.372676 4581 leader_election.cc:288] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 14:42:35.372772 4585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 14:42:35.372987 4585 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [term 3 LEADER]: Becoming Leader. State: Replica: 0d45fad648bc448893b80c7429c4ee52, State: Running, Role: LEADER
I20260808 14:42:35.373307 4585 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } }
I20260808 14:42:35.374053 4581 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:42:35.374325 4585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d45fad648bc448893b80c7429c4ee52. Latest consensus state: current_term: 3 leader_uuid: "0d45fad648bc448893b80c7429c4ee52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } } }
I20260808 14:42:35.374447 4585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [sys.catalog]: This master's current role is: LEADER
I20260808 14:42:35.374877 4589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:42:35.375030 4586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0d45fad648bc448893b80c7429c4ee52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d45fad648bc448893b80c7429c4ee52" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35049 } } }
I20260808 14:42:35.375263 4586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52 [sys.catalog]: This master's current role is: LEADER
I20260808 14:42:35.380426 4589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:42:35.381374 4589 catalog_manager.cc:1259] Loaded cluster ID: 1b214b6b360142ca8c5cec785f370806
I20260808 14:42:35.381444 4589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:42:35.384054 4589 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:42:35.384874 4589 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Loaded TSK: 0
I20260808 14:42:35.384969 4589 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0d45fad648bc448893b80c7429c4ee52: Loaded TSK: 1
F20260808 14:42:35.385826 4589 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f81eea369ad google::LogMessage::Fail() at ??:0
@ 0x7f81eea3988a google::LogMessage::SendToLog() at ??:0
@ 0x7f81eea364de google::LogMessage::Flush() at ??:0
@ 0x7f81eea37a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f81f16fa18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f81f16f2f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f81f939b7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f81f937599c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f81f9376178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f81f93b7df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f81f93d682b std::function<>::operator()() at ??:0
@ 0x7f81f9375b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f81f9376f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f81f9373579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f81f93b7640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f81f0c883d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f81f0c88c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f81f0c8a3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f81f0c79512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f81f22bd6db start_thread at ??:0
@ 0x7f81ed53461f clone at ??:0
W20260808 14:42:35.468998 4583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:35.469273 4583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:35.469316 4583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:35.474566 4583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:35.474627 4583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:35.474691 4583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:42:35.481163 4583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:41059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=44069
--tserver_master_addrs=127.19.144.126:35049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:35.482405 4583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:35.483448 4583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:35.494295 4627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:35.494295 4625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:35.494484 4624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:35.494736 4583 server_base.cc:1034] running on GCE node
I20260808 14:42:35.495348 4583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:35.496270 4583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:35.497504 4583 hybrid_clock.cc:648] HybridClock initialized: now 1786200155497475 us; error 47 us; skew 500 ppm
Aug 08 14:42:35 dist-test-slave-p3t5 krb5kdc[3411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200155, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:35.501667 4583 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:42:35.503026 4583 webserver.cc:460] Webserver started at http://127.19.144.65:44069/ using document root <none> and password file <none>
I20260808 14:42:35.503959 4583 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:35.504060 4583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:35.510082 4583 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:42:35.512674 4638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:35.514195 4583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:42:35.514355 4583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "0e1e723868bc4464832909357429ad27"
format_stamp: "Formatted at 2026-08-08 14:41:53 on dist-test-slave-p3t5"
server_key: "8c0fafcd93394c51ec44b13fcabd6597"
server_key_iv: "400ab80e369ded9b435fc00c96eaa1f5"
server_key_version: "encryptionkey@0"
I20260808 14:42:35.514954 4583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:35.543462 4583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:35.547204 4583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:35.547449 4583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:35.548128 4583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:35.549373 4646 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 14:42:35.555591 4583 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 14:42:35.555727 4583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.000s
I20260808 14:42:35.555817 4583 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 14:42:35.557901 4583 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 14:42:35.557991 4583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260808 14:42:35.558320 4646 tablet_bootstrap.cc:492] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27: Bootstrap starting.
I20260808 14:42:35.569458 4583 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:41059
I20260808 14:42:35.569789 4758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:41059 every 8 connection(s)
I20260808 14:42:35.570677 4583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:42:35.577004 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 4583
I20260808 14:42:35.577533 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:34323
--local_ip_for_outbound_sockets=127.19.144.66
--tserver_master_addrs=127.19.144.126:35049
--webserver_port=45287
--webserver_interface=127.19.144.66
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:42:35.581379 4646 log.cc:824] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:35.591409 4646 tablet_bootstrap.cc:492] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 14:42:35.591933 4646 tablet_bootstrap.cc:492] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27: Bootstrap complete.
I20260808 14:42:35.593425 4646 ts_tablet_manager.cc:1397] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27: Time spent bootstrapping tablet: real 0.035s user 0.027s sys 0.004s
I20260808 14:42:35.596988 4646 raft_consensus.cc:348] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:35.597661 4646 raft_consensus.cc:729] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1e723868bc4464832909357429ad27, State: Initialized, Role: FOLLOWER
I20260808 14:42:35.598311 4646 consensus_queue.cc:260] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:35.599458 4646 ts_tablet_manager.cc:1428] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27: Time spent starting tablet: real 0.006s user 0.005s sys 0.004s
W20260808 14:42:35.646752 4763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:42:35.572949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:42927)
0808 14:42:35.573488 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:35.573549 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:35.574468 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:35.645685 (+ 71217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":151,"threads_started":1}
W20260808 14:42:35.647383 4761 heartbeater.cc:644] Failed to heartbeat to 127.19.144.126:35049 (0 consecutive failures): Network error: Failed to ping master at 127.19.144.126:35049: Client connection negotiation failed: client connection to 127.19.144.126:35049: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 14:42:35.648231 4763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:35.647909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:41769)
0808 14:42:35.648064 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:35.648149 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 14:42:35.649004 4763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:35.648741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:33217)
0808 14:42:35.648890 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:35.648945 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260808 14:42:35.649221 4761 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 14:42:35.700520 4764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:35.700811 4764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:35.700860 4764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:35.705034 4764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:35.705101 4764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:35.705163 4764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:42:35.710686 4764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:34323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=45287
--tserver_master_addrs=127.19.144.126:35049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:35.711887 4764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:35.713001 4764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:35.723415 4775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:35.723415 4772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:35.723445 4773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:35.723786 4764 server_base.cc:1034] running on GCE node
I20260808 14:42:35.724274 4764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:35.725013 4764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:35.726207 4764 hybrid_clock.cc:648] HybridClock initialized: now 1786200155726189 us; error 31 us; skew 500 ppm
Aug 08 14:42:35 dist-test-slave-p3t5 krb5kdc[3411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200155, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:35.729302 4764 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:42:35.730306 4764 webserver.cc:460] Webserver started at http://127.19.144.66:45287/ using document root <none> and password file <none>
I20260808 14:42:35.730917 4764 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:35.730983 4764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:35.734999 4764 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:42:35.737342 4782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:35.738781 4764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:42:35.738915 4764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "1466ce81810245f099fcf8dfa47444d2"
format_stamp: "Formatted at 2026-08-08 14:41:53 on dist-test-slave-p3t5"
server_key: "c188e8471ab5d02c4986db89502f2be6"
server_key_iv: "c0806f8b9eac0b4c34716cef43e51b39"
server_key_version: "encryptionkey@0"
I20260808 14:42:35.739506 4764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:35.752095 4764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:35.755831 4764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:35.756065 4764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:35.756762 4764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:35.757974 4764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:42:35.758036 4764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:35.758108 4764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:42:35.758147 4764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:35.767984 4764 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:34323
I20260808 14:42:35.768016 4899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:34323 every 8 connection(s)
I20260808 14:42:35.769104 4764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:42:35.773350 4902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:35.771502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:47801)
0808 14:42:35.772009 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:35.772130 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":119,"threads_started":1}
W20260808 14:42:35.773854 4900 heartbeater.cc:644] Failed to heartbeat to 127.19.144.126:35049 (0 consecutive failures): Network error: Failed to ping master at 127.19.144.126:35049: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
I20260808 14:42:35.774537 4902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:35.774294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:52055)
0808 14:42:35.774415 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:35.774481 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 14:42:35.775157 4902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:35.774942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:51459)
0808 14:42:35.775051 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:35.775099 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 14:42:35.775336 4900 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 14:42:35.776093 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 4764
I20260808 14:42:35.776541 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:36605
--local_ip_for_outbound_sockets=127.19.144.67
--tserver_master_addrs=127.19.144.126:35049
--webserver_port=33425
--webserver_interface=127.19.144.67
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:42:35.896121 4903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:35.896420 4903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:35.896474 4903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:35.900765 4903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:35.900854 4903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:35.900918 4903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:42:35.906863 4903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:36605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=33425
--tserver_master_addrs=127.19.144.126:35049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:35.908082 4903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:35.909204 4903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:35.919916 4908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:35.920002 4911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:35.920083 4903 server_base.cc:1034] running on GCE node
W20260808 14:42:35.920249 4909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:35.920706 4903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:35.921434 4903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:35.922605 4903 hybrid_clock.cc:648] HybridClock initialized: now 1786200155922585 us; error 43 us; skew 500 ppm
Aug 08 14:42:35 dist-test-slave-p3t5 krb5kdc[3411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200155, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:35.925566 4903 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:42:35.926667 4903 webserver.cc:460] Webserver started at http://127.19.144.67:33425/ using document root <none> and password file <none>
I20260808 14:42:35.927301 4903 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:35.927413 4903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:35.931639 4903 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 14:42:35.934024 4918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:35.935405 4903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:42:35.935586 4903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "a93271875de240889bb59007953575ce"
format_stamp: "Formatted at 2026-08-08 14:41:53 on dist-test-slave-p3t5"
server_key: "d1dd45421e15d389754beff2fd012c23"
server_key_iv: "7fb0f886192448f646d72c137cbdb23f"
server_key_version: "encryptionkey@0"
I20260808 14:42:35.936220 4903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:35.947957 4903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:35.951539 4903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:35.951797 4903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:35.952491 4903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:35.953408 4903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:42:35.953481 4903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:35.953554 4903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:42:35.953608 4903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:35.963655 4903 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:36605
I20260808 14:42:35.963677 5035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:36605 every 8 connection(s)
I20260808 14:42:35.964809 4903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:42:35.969008 5038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:35.967098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:39293)
0808 14:42:35.967627 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:35.967755 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":122,"threads_started":1}
W20260808 14:42:35.969530 5036 heartbeater.cc:644] Failed to heartbeat to 127.19.144.126:35049 (0 consecutive failures): Network error: Failed to ping master at 127.19.144.126:35049: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
I20260808 14:42:35.970217 5038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:35.969983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:45205)
0808 14:42:35.970106 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:35.970158 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 14:42:35.970901 5038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:35.970694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:46095)
0808 14:42:35.970803 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:35.970849 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 14:42:35.971108 5036 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 14:42:35.974718 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 4903
I20260808 14:42:36.650522 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:36.650001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:48131)
0808 14:42:36.650317 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:36.650395 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":87,"threads_started":1}
I20260808 14:42:36.776427 5041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:36.775921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:36543)
0808 14:42:36.776236 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:36.776304 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":94,"threads_started":1}
I20260808 14:42:36.972313 5042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:36.971815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:37899)
0808 14:42:36.972112 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:36.972174 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":119,"threads_started":1}
I20260808 14:42:37.128078 5043 raft_consensus.cc:482] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:42:37.128386 5043 raft_consensus.cc:504] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:37.129670 5043 leader_election.cc:288] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a93271875de240889bb59007953575ce (127.19.144.67:36605), 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66:34323)
Aug 08 14:42:37 dist-test-slave-p3t5 krb5kdc[3411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200155, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.66@KRBTEST.COM
Aug 08 14:42:37 dist-test-slave-p3t5 krb5kdc[3411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200155, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.67@KRBTEST.COM
I20260808 14:42:37.144742 5046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:37.130311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:37231 (local address 127.19.144.66:34323)
0808 14:42:37.130611 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:37.130637 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:37.130675 (+ 38us) server_negotiation.cc:407] Connection header received
0808 14:42:37.131356 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:37.131362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:37.131713 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:37.132029 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:37.133447 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:37.134508 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:37.135833 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:37.136285 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:37.136487 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:37.139423 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:37.139430 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:37.139448 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:37.139509 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:37.141780 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:37.142434 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:37.142446 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:37.142451 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:37.142575 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:37.142986 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:37.142991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:37.142992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:37.143500 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:37.143662 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:37.144228 (+ 566us) server_negotiation.cc:299] Negotiation successful
0808 14:42:37.144548 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":100,"threads_started":1}
I20260808 14:42:37.144747 5045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:37.130048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:55285 (local address 127.19.144.67:36605)
0808 14:42:37.130630 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:37.130651 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:37.130684 (+ 33us) server_negotiation.cc:407] Connection header received
0808 14:42:37.131356 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:37.131362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:37.131711 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:37.131969 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:37.133451 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:37.134494 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:37.135802 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:37.136286 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:37.136470 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:37.139817 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:37.139822 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:37.139833 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:37.139863 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:37.141754 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:37.142405 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:37.142412 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:37.142418 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:37.142553 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:37.143130 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:37.143134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:37.143137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:37.143526 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:37.143674 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:37.144227 (+ 553us) server_negotiation.cc:299] Negotiation successful
0808 14:42:37.144533 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":120,"threads_started":1}
I20260808 14:42:37.144733 5044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:37.129925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:34323 (local address 127.19.144.65:37231)
0808 14:42:37.130271 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:37.130285 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:37.130399 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:37.132237 (+ 1838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:37.132241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:37.132716 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:37.133255 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:37.133271 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:37.134678 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:37.134680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:37.135579 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:37.135590 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:37.136420 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:37.136424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:37.136646 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:37.137405 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:42:37.137418 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:37.139155 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:37.141914 (+ 2759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:37.141916 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:37.141918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:37.142294 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:37.142711 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:37.142716 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:37.142722 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:37.142859 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:37.143633 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:37.143640 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:37.143913 (+ 273us) client_negotiation.cc:769] Sending connection context
0808 14:42:37.144096 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 14:42:37.144439 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":134,"spinlock_wait_cycles":40576,"thread_start_us":72,"threads_started":1}
I20260808 14:42:37.144734 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:37.129847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:36605 (local address 127.19.144.65:55285)
0808 14:42:37.129998 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:37.130016 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:37.130134 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:37.132108 (+ 1974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:37.132115 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:37.132717 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:37.133255 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:37.133271 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:37.134647 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:37.134652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:37.135578 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:37.135589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:37.136423 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:37.136427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:37.136670 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:37.137352 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:42:37.137369 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:37.139589 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:37.141877 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:37.141885 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:37.141903 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:37.142283 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:37.142683 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:37.142689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:37.142693 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:37.142869 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:37.143640 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:37.143643 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:37.143903 (+ 260us) client_negotiation.cc:769] Sending connection context
0808 14:42:37.144096 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 14:42:37.144533 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":306,"spinlock_wait_cycles":14592}
W20260808 14:42:37.147698 4639 leader_election.cc:341] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a93271875de240889bb59007953575ce (127.19.144.67:36605): Not found: Tablet not found: 57764198e4ec482fb293c62c80036d1a
W20260808 14:42:37.147984 4640 leader_election.cc:341] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66:34323): Not found: Tablet not found: 57764198e4ec482fb293c62c80036d1a
I20260808 14:42:37.148104 4640 leader_election.cc:302] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e1e723868bc4464832909357429ad27; no voters: 1466ce81810245f099fcf8dfa47444d2, a93271875de240889bb59007953575ce
I20260808 14:42:37.148401 5043 raft_consensus.cc:2726] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:42:37.651822 5047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:37.651391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:46173)
0808 14:42:37.651660 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:37.651720 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":91,"threads_started":1}
I20260808 14:42:37.777989 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:37.777361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:45011)
0808 14:42:37.777749 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:37.777842 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":86,"threads_started":1}
I20260808 14:42:37.973791 5049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:37.973203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:44697)
0808 14:42:37.973495 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:37.973568 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":100,"threads_started":1}
I20260808 14:42:38.653229 5050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:38.652716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:43637)
0808 14:42:38.653055 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:38.653124 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":119,"threads_started":1}
I20260808 14:42:38.779345 5051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:38.778900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:37079)
0808 14:42:38.779185 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:38.779242 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":94,"threads_started":1}
I20260808 14:42:38.975289 5052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:38.974713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:56187)
0808 14:42:38.975102 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:38.975182 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":152,"threads_started":1}
I20260808 14:42:39.362433 5053 raft_consensus.cc:482] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:42:39.362617 5053 raft_consensus.cc:504] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:39.363119 5053 leader_election.cc:288] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a93271875de240889bb59007953575ce (127.19.144.67:36605), 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66:34323)
W20260808 14:42:39.364008 4639 leader_election.cc:341] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a93271875de240889bb59007953575ce (127.19.144.67:36605): Not found: Tablet not found: 57764198e4ec482fb293c62c80036d1a
W20260808 14:42:39.364172 4640 leader_election.cc:341] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66:34323): Not found: Tablet not found: 57764198e4ec482fb293c62c80036d1a
I20260808 14:42:39.364271 4640 leader_election.cc:302] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e1e723868bc4464832909357429ad27; no voters: 1466ce81810245f099fcf8dfa47444d2, a93271875de240889bb59007953575ce
I20260808 14:42:39.364475 5053 raft_consensus.cc:2726] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:42:39.654874 5054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:39.654281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:36055)
0808 14:42:39.654645 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:39.654735 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":105,"threads_started":1}
I20260808 14:42:39.780830 5055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:39.780344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:41823)
0808 14:42:39.780650 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:39.780719 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":94,"threads_started":1}
I20260808 14:42:39.976672 5056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:39.976207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:36877)
0808 14:42:39.976515 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:39.976572 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":99,"threads_started":1}
I20260808 14:42:40.656456 5057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:40.655998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:47523)
0808 14:42:40.656297 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:40.656357 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":80,"threads_started":1}
I20260808 14:42:40.782455 5058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:40.781964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:55709)
0808 14:42:40.782271 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:40.782341 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":90,"threads_started":1}
I20260808 14:42:40.977995 5059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:40.977512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:43167)
0808 14:42:40.977826 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:40.977891 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":147,"threads_started":1}
I20260808 14:42:41.657960 5060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:41.657366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:45715)
0808 14:42:41.657745 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:41.657846 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":104,"threads_started":1}
I20260808 14:42:41.783898 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:41.783405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:37625)
0808 14:42:41.783714 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:41.783790 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":88,"threads_started":1}
I20260808 14:42:41.979355 5062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:41.978842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:33869)
0808 14:42:41.979181 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:41.979250 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":90,"threads_started":1}
I20260808 14:42:42.659480 5063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:42.658966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:59867)
0808 14:42:42.659288 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:42.659359 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":87,"threads_started":1}
I20260808 14:42:42.723482 5064 raft_consensus.cc:482] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:42:42.723634 5064 raft_consensus.cc:504] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:42.724143 5064 leader_election.cc:288] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a93271875de240889bb59007953575ce (127.19.144.67:36605), 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66:34323)
W20260808 14:42:42.725114 4640 leader_election.cc:341] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66:34323): Not found: Tablet not found: 57764198e4ec482fb293c62c80036d1a
W20260808 14:42:42.725260 4639 leader_election.cc:341] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a93271875de240889bb59007953575ce (127.19.144.67:36605): Not found: Tablet not found: 57764198e4ec482fb293c62c80036d1a
I20260808 14:42:42.725353 4639 leader_election.cc:302] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e1e723868bc4464832909357429ad27; no voters: 1466ce81810245f099fcf8dfa47444d2, a93271875de240889bb59007953575ce
I20260808 14:42:42.725526 5064 raft_consensus.cc:2726] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:42:42.785189 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:42.784689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:57927)
0808 14:42:42.785020 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:42.785085 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":98,"threads_started":1}
I20260808 14:42:42.980840 5067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:42.980310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:34117)
0808 14:42:42.980668 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:42.980736 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":105,"threads_started":1}
I20260808 14:42:43.661131 5068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:43.660480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:36699)
0808 14:42:43.660904 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:43.660992 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":104,"threads_started":1}
I20260808 14:42:43.786635 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:43.786159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:57517)
0808 14:42:43.786462 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:43.786533 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":107,"threads_started":1}
I20260808 14:42:43.982438 5070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:43.981938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:38071)
0808 14:42:43.982247 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:43.982330 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":85,"threads_started":1}
I20260808 14:42:44.662503 5071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:44.662051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:53795)
0808 14:42:44.662327 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:44.662387 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":78,"threads_started":1}
I20260808 14:42:44.788166 5072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:44.787654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:40779)
0808 14:42:44.787994 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:44.788061 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
I20260808 14:42:44.983783 5073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:44.983303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:47195)
0808 14:42:44.983618 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:44.983682 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":101,"threads_started":1}
I20260808 14:42:45.663980 5074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:45.663512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:39133)
0808 14:42:45.663815 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:45.663877 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":102,"threads_started":1}
I20260808 14:42:45.789601 5075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:45.789099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:34709)
0808 14:42:45.789420 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:45.789495 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":87,"threads_started":1}
I20260808 14:42:45.898518 5076 raft_consensus.cc:482] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:42:45.898676 5076 raft_consensus.cc:504] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:45.899251 5076 leader_election.cc:288] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a93271875de240889bb59007953575ce (127.19.144.67:36605), 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66:34323)
W20260808 14:42:45.900148 4639 leader_election.cc:341] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a93271875de240889bb59007953575ce (127.19.144.67:36605): Not found: Tablet not found: 57764198e4ec482fb293c62c80036d1a
W20260808 14:42:45.900277 4640 leader_election.cc:341] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66:34323): Not found: Tablet not found: 57764198e4ec482fb293c62c80036d1a
I20260808 14:42:45.900326 4639 leader_election.cc:302] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e1e723868bc4464832909357429ad27; no voters: 1466ce81810245f099fcf8dfa47444d2, a93271875de240889bb59007953575ce
I20260808 14:42:45.900503 5076 raft_consensus.cc:2726] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:42:45.985245 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:45.984732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:57923)
0808 14:42:45.985076 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:45.985141 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":139,"threads_started":1}
I20260808 14:42:46.665449 5078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:46.664947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:47733)
0808 14:42:46.665262 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:46.665333 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":94,"threads_started":1}
I20260808 14:42:46.791105 5079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:46.790632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:40285)
0808 14:42:46.790924 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:46.790991 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":78,"threads_started":1}
I20260808 14:42:46.986748 5080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:46.986242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:50787)
0808 14:42:46.986555 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:46.986622 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":86,"threads_started":1}
I20260808 14:42:47.666811 5081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:47.666352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:39055)
0808 14:42:47.666630 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:47.666693 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":85,"threads_started":1}
I20260808 14:42:47.792479 5082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:47.792034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:39825)
0808 14:42:47.792316 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:47.792372 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":88,"threads_started":1}
I20260808 14:42:47.988305 5083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:47.987813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:35645)
0808 14:42:47.988135 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:47.988199 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":124,"threads_started":1}
I20260808 14:42:48.668300 5084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:48.667775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:51239)
0808 14:42:48.668122 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:48.668191 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":129,"threads_started":1}
I20260808 14:42:48.793939 5085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:48.793369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:37569)
0808 14:42:48.793725 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:48.793829 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":96,"threads_started":1}
I20260808 14:42:48.989926 5086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:48.989278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:53593)
0808 14:42:48.989731 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:48.989808 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":126,"threads_started":1}
I20260808 14:42:49.669935 5087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:49.669393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:42705)
0808 14:42:49.669756 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:49.669826 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":91,"threads_started":1}
I20260808 14:42:49.795508 5088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:49.794923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:54665)
0808 14:42:49.795306 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:49.795391 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":100,"threads_started":1}
I20260808 14:42:49.925746 5089 raft_consensus.cc:482] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:42:49.925899 5089 raft_consensus.cc:504] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a93271875de240889bb59007953575ce" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 36605 } } peers { permanent_uuid: "1466ce81810245f099fcf8dfa47444d2" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 34323 } } peers { permanent_uuid: "0e1e723868bc4464832909357429ad27" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 41059 } }
I20260808 14:42:49.926487 5089 leader_election.cc:288] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a93271875de240889bb59007953575ce (127.19.144.67:36605), 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66:34323)
W20260808 14:42:49.927314 4639 leader_election.cc:341] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a93271875de240889bb59007953575ce (127.19.144.67:36605): Not found: Tablet not found: 57764198e4ec482fb293c62c80036d1a
W20260808 14:42:49.927421 4640 leader_election.cc:341] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1466ce81810245f099fcf8dfa47444d2 (127.19.144.66:34323): Not found: Tablet not found: 57764198e4ec482fb293c62c80036d1a
I20260808 14:42:49.927544 4640 leader_election.cc:302] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e1e723868bc4464832909357429ad27; no voters: 1466ce81810245f099fcf8dfa47444d2, a93271875de240889bb59007953575ce
I20260808 14:42:49.927721 5089 raft_consensus.cc:2726] T 57764198e4ec482fb293c62c80036d1a P 0e1e723868bc4464832909357429ad27 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 14:42:49.991601 5090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:49.990988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:42715)
0808 14:42:49.991384 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:49.991461 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":131,"threads_started":1}
I20260808 14:42:50.671456 5091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:50.670971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.65:39593)
0808 14:42:50.671276 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:50.671344 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":83,"threads_started":1}
I20260808 14:42:50.797190 5092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:50.796708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.66:57333)
0808 14:42:50.797018 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:50.797086 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":102,"threads_started":1}
I20260808 14:42:50.977824 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 4583
I20260808 14:42:50.987618 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 4764
I20260808 14:42:50.993330 5093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:50.992873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35049 (local address 127.19.144.67:48987)
0808 14:42:50.993160 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:50.993223 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.126:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":98,"threads_started":1}
I20260808 14:42:50.995604 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 4903
2026-08-08T14:42:50Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59583 ms)
[----------] 30 tests from SecurityITest (248641 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:42:51 dist-test-slave-p3t5 krb5kdc[5096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:42:51 dist-test-slave-p3t5 krb5kdc[5096](info): set up 2 sockets
Aug 08 14:42:51 dist-test-slave-p3t5 krb5kdc[5096](info): commencing operation
krb5kdc: starting...
W20260808 14:42:52.067823 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:42:52 dist-test-slave-p3t5 krb5kdc[5096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:42:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:42:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:42:52.209471 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:46463
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:46463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:42:52.326987 5110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:52.327360 5110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:52.327442 5110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:52.331650 5110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:42:52.331779 5110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:52.331820 5110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:52.331859 5110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:42:52.331892 5110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:42:52.337450 5110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:46463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:46463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:52.338750 5110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:52.339887 5110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:52.349421 5116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:52.349426 5115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:52.349426 5118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:52.349486 5110 server_base.cc:1034] running on GCE node
I20260808 14:42:52.350419 5110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:52.351713 5110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:52.352900 5110 hybrid_clock.cc:648] HybridClock initialized: now 1786200172352879 us; error 36 us; skew 500 ppm
Aug 08 14:42:52 dist-test-slave-p3t5 krb5kdc[5096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200172, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:52.356174 5110 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:42:52.357251 5110 webserver.cc:460] Webserver started at http://127.19.144.126:34315/ using document root <none> and password file <none>
I20260808 14:42:52.357956 5110 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:52.358027 5110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:52.358280 5110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:52.360509 5110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "0b3614509c3d4900b8fdb1fac7c1ea6b"
format_stamp: "Formatted at 2026-08-08 14:42:52 on dist-test-slave-p3t5"
server_key: "b9b44942b7f7998b3c926c0f8fdf6930"
server_key_iv: "521cba3539bcf52ec645e82d40b2359c"
server_key_version: "encryptionkey@0"
I20260808 14:42:52.361150 5110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "0b3614509c3d4900b8fdb1fac7c1ea6b"
format_stamp: "Formatted at 2026-08-08 14:42:52 on dist-test-slave-p3t5"
server_key: "b9b44942b7f7998b3c926c0f8fdf6930"
server_key_iv: "521cba3539bcf52ec645e82d40b2359c"
server_key_version: "encryptionkey@0"
I20260808 14:42:52.365492 5110 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:42:52.368242 5127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:52.369575 5110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:42:52.369784 5110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "0b3614509c3d4900b8fdb1fac7c1ea6b"
format_stamp: "Formatted at 2026-08-08 14:42:52 on dist-test-slave-p3t5"
server_key: "b9b44942b7f7998b3c926c0f8fdf6930"
server_key_iv: "521cba3539bcf52ec645e82d40b2359c"
server_key_version: "encryptionkey@0"
I20260808 14:42:52.369915 5110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:52.380857 5110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:52.384038 5110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:52.384270 5110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:52.392887 5110 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:46463
I20260808 14:42:52.392953 5180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:46463 every 8 connection(s)
I20260808 14:42:52.394045 5110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:42:52.397149 5181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:52.397183 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 5110
I20260808 14:42:52.397346 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:42:52.397727 20033 external_mini_cluster.cc:1428] Setting key 939e63689dddb3a116b84625a5f5431a
I20260808 14:42:52.405025 5181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b: Bootstrap starting.
I20260808 14:42:52.407709 5181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b: Neither blocks nor log segments found. Creating new log.
Aug 08 14:42:52 dist-test-slave-p3t5 krb5kdc[5096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:42:52.408713 5181 log.cc:824] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:52.410916 5181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b: No bootstrap required, opened a new log
I20260808 14:42:52.413731 5184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:52.399339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33404 (local address 127.19.144.126:46463)
0808 14:42:52.399851 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:52.399872 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:52.399920 (+ 48us) server_negotiation.cc:407] Connection header received
0808 14:42:52.400865 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:52.400893 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:52.401359 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:52.401807 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:52.402522 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:52.404091 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:52.404977 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:52.405695 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:52.405854 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:52.408748 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:52.408753 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:52.408768 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:52.408863 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:52.410742 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:52.411270 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:52.411276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:52.411281 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:52.411403 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:52.411711 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:52.411713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:52.411714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:52.412110 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:52.412261 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:52.412573 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 14:42:52.412896 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":142,"threads_started":1}
I20260808 14:42:52.415313 5181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3614509c3d4900b8fdb1fac7c1ea6b" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46463 } }
I20260808 14:42:52.415607 5181 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:52.415657 5181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b3614509c3d4900b8fdb1fac7c1ea6b, State: Initialized, Role: FOLLOWER
I20260808 14:42:52.416157 5181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3614509c3d4900b8fdb1fac7c1ea6b" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46463 } }
I20260808 14:42:52.416332 5181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:42:52.416388 5181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:42:52.416463 5181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:52.417418 5181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3614509c3d4900b8fdb1fac7c1ea6b" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46463 } }
I20260808 14:42:52.417814 5181 leader_election.cc:302] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b3614509c3d4900b8fdb1fac7c1ea6b; no voters:
I20260808 14:42:52.418244 5181 leader_election.cc:288] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:42:52.418383 5186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:42:52.418649 5186 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [term 1 LEADER]: Becoming Leader. State: Replica: 0b3614509c3d4900b8fdb1fac7c1ea6b, State: Running, Role: LEADER
I20260808 14:42:52.419024 5186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3614509c3d4900b8fdb1fac7c1ea6b" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46463 } }
I20260808 14:42:52.419633 5181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:42:52.420910 5187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b3614509c3d4900b8fdb1fac7c1ea6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3614509c3d4900b8fdb1fac7c1ea6b" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46463 } } }
I20260808 14:42:52.420874 5188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b3614509c3d4900b8fdb1fac7c1ea6b. Latest consensus state: current_term: 1 leader_uuid: "0b3614509c3d4900b8fdb1fac7c1ea6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3614509c3d4900b8fdb1fac7c1ea6b" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46463 } } }
I20260808 14:42:52.421034 5188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [sys.catalog]: This master's current role is: LEADER
I20260808 14:42:52.421034 5187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b [sys.catalog]: This master's current role is: LEADER
I20260808 14:42:52.421667 5195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:42:52.424508 5195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:42:52.429828 5195 catalog_manager.cc:1347] Generated new cluster ID: e970e745bc1b425f8a0f1e6346c0157a
I20260808 14:42:52.429899 5195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:42:52.441365 5195 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:42:52.442813 5195 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:42:52.454192 5195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b: Generated new TSK 0
I20260808 14:42:52.455188 5195 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:42:52.497761 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46463
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:42:52.623095 5207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:52.623415 5207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:52.623464 5207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:52.627866 5207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:52.627948 5207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:52.628010 5207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:42:52.633641 5207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:52.634864 5207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:52.636034 5207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:52.646235 5213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:52.646283 5212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:52.646262 5215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:52.646498 5207 server_base.cc:1034] running on GCE node
I20260808 14:42:52.646935 5207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:52.647671 5207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:52.648880 5207 hybrid_clock.cc:648] HybridClock initialized: now 1786200172648853 us; error 36 us; skew 500 ppm
Aug 08 14:42:52 dist-test-slave-p3t5 krb5kdc[5096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200172, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:52.652308 5207 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:42:52.653367 5207 webserver.cc:460] Webserver started at http://127.19.144.65:43859/ using document root <none> and password file <none>
I20260808 14:42:52.654019 5207 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:52.654086 5207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:52.654268 5207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:52.656520 5207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "05dadbdcb7924e68bccb5ae281343377"
format_stamp: "Formatted at 2026-08-08 14:42:52 on dist-test-slave-p3t5"
server_key: "312941bef9bfbfca571bede3a5f687f2"
server_key_iv: "28fa16056a270a27ffeef532b55ba8b6"
server_key_version: "encryptionkey@0"
I20260808 14:42:52.657078 5207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "05dadbdcb7924e68bccb5ae281343377"
format_stamp: "Formatted at 2026-08-08 14:42:52 on dist-test-slave-p3t5"
server_key: "312941bef9bfbfca571bede3a5f687f2"
server_key_iv: "28fa16056a270a27ffeef532b55ba8b6"
server_key_version: "encryptionkey@0"
I20260808 14:42:52.661017 5207 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:42:52.663655 5222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:52.664983 5207 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 14:42:52.665089 5207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "05dadbdcb7924e68bccb5ae281343377"
format_stamp: "Formatted at 2026-08-08 14:42:52 on dist-test-slave-p3t5"
server_key: "312941bef9bfbfca571bede3a5f687f2"
server_key_iv: "28fa16056a270a27ffeef532b55ba8b6"
server_key_version: "encryptionkey@0"
I20260808 14:42:52.665184 5207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:52.686017 5207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:52.689325 5207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:52.689492 5207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:52.690155 5207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:52.690996 5207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:42:52.691047 5207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:52.691089 5207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:42:52.691104 5207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:52.700383 5207 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:38961
I20260808 14:42:52.700404 5338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:38961 every 8 connection(s)
I20260808 14:42:52.701341 5207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:42:52.706454 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 5207
I20260808 14:42:52.706621 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:42:52.706928 20033 external_mini_cluster.cc:1428] Setting key 1b036b94d39595e07d31c7c98fdcadd8
Aug 08 14:42:52 dist-test-slave-p3t5 krb5kdc[5096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200172, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:42:52.717155 5184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:52.703350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:51851 (local address 127.19.144.126:46463)
0808 14:42:52.703515 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:52.703520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:52.704500 (+ 980us) server_negotiation.cc:407] Connection header received
0808 14:42:52.705894 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:52.705897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:52.705948 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:52.706026 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:52.707450 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:52.708129 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:52.709798 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:52.710064 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:52.710189 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:52.713198 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:52.713202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:52.713204 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:52.713230 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:52.714885 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:52.715488 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:52.715491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:52.715492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:52.715576 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:52.715948 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:52.715953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:52.715955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:52.716160 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:52.716277 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:52.716833 (+ 556us) server_negotiation.cc:299] Negotiation successful
0808 14:42:52.716981 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:42:52.718212 5341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:52.703828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46463 (local address 127.19.144.65:51851)
0808 14:42:52.704324 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:52.704370 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:52.705660 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:52.706190 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:52.706198 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:52.706567 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:52.707253 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:52.707268 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:52.708315 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:52.708321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:52.709596 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:52.709608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:52.710245 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:52.710250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:52.710436 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:52.711087 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:42:52.711106 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:52.712999 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:52.715032 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:52.715038 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:52.715058 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:52.715359 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:52.715676 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:52.715679 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:52.715681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:52.715844 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:52.716284 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:52.716294 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:52.716577 (+ 283us) client_negotiation.cc:769] Sending connection context
0808 14:42:52.716784 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 14:42:52.717155 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":129,"threads_started":1}
I20260808 14:42:52.719632 5339 heartbeater.cc:347] Connected to a master server at 127.19.144.126:46463
I20260808 14:42:52.719975 5339 heartbeater.cc:464] Registering TS with master...
I20260808 14:42:52.720641 5339 heartbeater.cc:511] Master 127.19.144.126:46463 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
I20260808 14:42:52.722515 5136 ts_manager.cc:194] Registered new tserver with Master: 05dadbdcb7924e68bccb5ae281343377 (127.19.144.65:38961)
I20260808 14:42:52.723727 5136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:51851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:42:52.741807 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46463
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:42:52.862941 5344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:52.863272 5344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:52.863320 5344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:52.867913 5344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:52.867995 5344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:52.868111 5344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:42:52.874090 5344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:52.875286 5344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:52.876346 5344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:52.886762 5349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:52.886752 5353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:52.886752 5351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:52.887176 5344 server_base.cc:1034] running on GCE node
I20260808 14:42:52.887627 5344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:52.888253 5344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:52.889461 5344 hybrid_clock.cc:648] HybridClock initialized: now 1786200172889436 us; error 30 us; skew 500 ppm
Aug 08 14:42:52 dist-test-slave-p3t5 krb5kdc[5096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200172, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:52.892581 5344 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:42:52.893482 5344 webserver.cc:460] Webserver started at http://127.19.144.66:35765/ using document root <none> and password file <none>
I20260808 14:42:52.894124 5344 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:52.894191 5344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:52.894367 5344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:52.896513 5344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "55884e0dbe4e488b9e6334afa27d98e5"
format_stamp: "Formatted at 2026-08-08 14:42:52 on dist-test-slave-p3t5"
server_key: "a5dcb87d8f0f0e1786d06bcff0061afc"
server_key_iv: "0ecddb9c03b7c0a4da62b902644e1091"
server_key_version: "encryptionkey@0"
I20260808 14:42:52.897085 5344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "55884e0dbe4e488b9e6334afa27d98e5"
format_stamp: "Formatted at 2026-08-08 14:42:52 on dist-test-slave-p3t5"
server_key: "a5dcb87d8f0f0e1786d06bcff0061afc"
server_key_iv: "0ecddb9c03b7c0a4da62b902644e1091"
server_key_version: "encryptionkey@0"
I20260808 14:42:52.901183 5344 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:42:52.903895 5361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:52.905264 5344 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:42:52.905406 5344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "55884e0dbe4e488b9e6334afa27d98e5"
format_stamp: "Formatted at 2026-08-08 14:42:52 on dist-test-slave-p3t5"
server_key: "a5dcb87d8f0f0e1786d06bcff0061afc"
server_key_iv: "0ecddb9c03b7c0a4da62b902644e1091"
server_key_version: "encryptionkey@0"
I20260808 14:42:52.905531 5344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:52.932463 5344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:52.936269 5344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:52.936506 5344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:52.937183 5344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:52.938176 5344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:42:52.938227 5344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:52.938300 5344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:42:52.938341 5344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:52.947883 5344 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:36911
I20260808 14:42:52.947911 5477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:36911 every 8 connection(s)
I20260808 14:42:52.948882 5344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:42:52.950639 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 5344
I20260808 14:42:52.950783 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:42:52.951148 20033 external_mini_cluster.cc:1428] Setting key 8ff69257a525243dacfa41e5da2c30d6
Aug 08 14:42:52 dist-test-slave-p3t5 krb5kdc[5096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200172, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
I20260808 14:42:52.965138 5184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:52.951044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:53333 (local address 127.19.144.126:46463)
0808 14:42:52.951216 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:52.951222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:52.952826 (+ 1604us) server_negotiation.cc:407] Connection header received
0808 14:42:52.954031 (+ 1205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:52.954035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:52.954088 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:52.954194 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:52.955639 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:52.956302 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:52.957557 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:52.957839 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:52.957926 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:52.961115 (+ 3189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:52.961119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:52.961121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:52.961148 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:52.962734 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:52.963331 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:52.963339 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:52.963340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:52.963436 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:52.963819 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:52.963822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:52.963824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:52.964119 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:52.964237 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:52.964813 (+ 576us) server_negotiation.cc:299] Negotiation successful
0808 14:42:52.964965 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:42:52.966142 5480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:52.951560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46463 (local address 127.19.144.66:53333)
0808 14:42:52.952627 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:52.952713 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:52.953786 (+ 1073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:52.954387 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:52.954394 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:52.954770 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:52.955465 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:52.955487 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:52.956450 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:52.956456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:52.957437 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:52.957448 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:52.958012 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:52.958019 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:52.958276 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:52.958996 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:42:52.959023 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:52.960893 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:52.962871 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:52.962878 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:52.962898 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:52.963202 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:52.963540 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:52.963543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:52.963545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:52.963711 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:52.964250 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:52.964259 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:52.964552 (+ 293us) client_negotiation.cc:769] Sending connection context
0808 14:42:52.964766 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 14:42:52.965157 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"thread_start_us":692,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
I20260808 14:42:52.967571 5478 heartbeater.cc:347] Connected to a master server at 127.19.144.126:46463
I20260808 14:42:52.967866 5478 heartbeater.cc:464] Registering TS with master...
I20260808 14:42:52.968437 5478 heartbeater.cc:511] Master 127.19.144.126:46463 requested a full tablet report, sending...
I20260808 14:42:52.969887 5136 ts_manager.cc:194] Registered new tserver with Master: 55884e0dbe4e488b9e6334afa27d98e5 (127.19.144.66:36911)
I20260808 14:42:52.970517 5136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:53333
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:42:52.985967 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46463
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:42:53.106897 5484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:53.107205 5484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:53.107254 5484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:53.111621 5484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:53.111702 5484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:53.111801 5484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:42:53.117452 5484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:46463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:53.118681 5484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:53.119820 5484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:53.133683 5492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:53.133719 5489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:53.133706 5490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:53.134232 5484 server_base.cc:1034] running on GCE node
I20260808 14:42:53.134748 5484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:53.135486 5484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:53.136700 5484 hybrid_clock.cc:648] HybridClock initialized: now 1786200173136688 us; error 37 us; skew 500 ppm
Aug 08 14:42:53 dist-test-slave-p3t5 krb5kdc[5096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200173, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:53.140125 5484 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:42:53.141208 5484 webserver.cc:460] Webserver started at http://127.19.144.67:41823/ using document root <none> and password file <none>
I20260808 14:42:53.141930 5484 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:53.142033 5484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:53.142290 5484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:53.144528 5484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "1e2c4a92541a4b3eb9ea0378ddbdc6e6"
format_stamp: "Formatted at 2026-08-08 14:42:53 on dist-test-slave-p3t5"
server_key: "a7d4ece58cd12863df744d8b5686a8f4"
server_key_iv: "7804b230718251c44750ed7862e16efc"
server_key_version: "encryptionkey@0"
I20260808 14:42:53.145159 5484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "1e2c4a92541a4b3eb9ea0378ddbdc6e6"
format_stamp: "Formatted at 2026-08-08 14:42:53 on dist-test-slave-p3t5"
server_key: "a7d4ece58cd12863df744d8b5686a8f4"
server_key_iv: "7804b230718251c44750ed7862e16efc"
server_key_version: "encryptionkey@0"
I20260808 14:42:53.149320 5484 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 14:42:53.152112 5499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:53.153447 5484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 14:42:53.153707 5484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "1e2c4a92541a4b3eb9ea0378ddbdc6e6"
format_stamp: "Formatted at 2026-08-08 14:42:53 on dist-test-slave-p3t5"
server_key: "a7d4ece58cd12863df744d8b5686a8f4"
server_key_iv: "7804b230718251c44750ed7862e16efc"
server_key_version: "encryptionkey@0"
I20260808 14:42:53.153855 5484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:53.164865 5484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:53.168457 5484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:53.168681 5484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:53.169386 5484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:53.170444 5484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:42:53.170518 5484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:53.170580 5484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:42:53.170631 5484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:53.180181 5484 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:42729
I20260808 14:42:53.180199 5619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:42729 every 8 connection(s)
I20260808 14:42:53.181167 5484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:42:53.184422 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 5484
I20260808 14:42:53.184590 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:42:53.184954 20033 external_mini_cluster.cc:1428] Setting key 8dfec6cfa6fb0249f55e67a17cac82de
Aug 08 14:42:53 dist-test-slave-p3t5 krb5kdc[5096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200173, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:42:53.196141 5184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.183117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:40531 (local address 127.19.144.126:46463)
0808 14:42:53.183276 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:53.183280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:53.184162 (+ 882us) server_negotiation.cc:407] Connection header received
0808 14:42:53.185965 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:53.185969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:53.186061 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:53.186169 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:53.187740 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.188266 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.189437 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.189838 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.189933 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.192636 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:53.192640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:53.192642 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:53.192668 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:53.194211 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:53.194720 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:53.194722 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:53.194723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:53.194804 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:53.195113 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:53.195115 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:53.195116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:53.195311 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:53.195388 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:53.195897 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 14:42:53.196010 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:42:53.197039 5622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.183503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:46463 (local address 127.19.144.67:40531)
0808 14:42:53.183993 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:53.184037 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:53.185698 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:53.186414 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:53.186424 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:53.186980 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:53.187563 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:53.187584 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.188379 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.188381 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:53.189295 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:53.189304 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.190010 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.190013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:53.190166 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.190782 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:42:53.190798 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:53.192394 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:53.194338 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:53.194343 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:53.194357 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:53.194630 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:53.194883 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:53.194885 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:53.194886 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:53.195032 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:53.195400 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:53.195405 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:53.195658 (+ 253us) client_negotiation.cc:769] Sending connection context
0808 14:42:53.195822 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 14:42:53.196115 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":123,"threads_started":1}
I20260808 14:42:53.198374 5620 heartbeater.cc:347] Connected to a master server at 127.19.144.126:46463
I20260808 14:42:53.198663 5620 heartbeater.cc:464] Registering TS with master...
I20260808 14:42:53.199314 5620 heartbeater.cc:511] Master 127.19.144.126:46463 requested a full tablet report, sending...
I20260808 14:42:53.200533 5136 ts_manager.cc:194] Registered new tserver with Master: 1e2c4a92541a4b3eb9ea0378ddbdc6e6 (127.19.144.67:42729)
I20260808 14:42:53.201090 5136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:40531
I20260808 14:42:53.212006 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:42:53.222263 5184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.214234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33406 (local address 127.19.144.126:46463)
0808 14:42:53.214402 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:53.214408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:53.214624 (+ 216us) server_negotiation.cc:407] Connection header received
0808 14:42:53.214711 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:53.214713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:53.214766 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:53.214847 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:53.215579 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.216201 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.217048 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.217275 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.217366 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.218861 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:53.218867 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:53.218874 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:53.218908 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:53.220480 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:53.220949 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:53.220952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:53.220953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:53.221038 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:53.221355 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:53.221357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:53.221358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:53.221597 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:53.221766 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:53.222005 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 14:42:53.222127 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 14:42:53.225482 5136 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:42:53.228826 5136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:42:53.244611 5632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.238267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38424 (local address 127.19.144.66:36911)
0808 14:42:53.238642 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:53.238655 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:53.238681 (+ 26us) server_negotiation.cc:407] Connection header received
0808 14:42:53.238895 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:53.238900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:53.239128 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:53.239301 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:53.239850 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.240960 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.242968 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.243877 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.244014 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.244184 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:53.244344 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 14:42:53.244450 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":122,"threads_started":1}
I20260808 14:42:53.244614 5633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.238898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:38961 (local address 127.0.0.1:41700)
0808 14:42:53.239161 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:53.239177 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:53.239275 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:53.240539 (+ 1264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:53.240542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:53.240562 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:53.240666 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:53.240671 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.241896 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.241900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:53.243097 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:53.243110 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.244078 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.244082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:53.244181 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.244207 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 14:42:53.244270 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 14:42:53.244417 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":55,"thread_start_us":130,"threads_started":1}
I20260808 14:42:53.244603 5631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.238073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:36911 (local address 127.0.0.1:38424)
0808 14:42:53.238494 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:53.238560 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:53.238703 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:53.239529 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:53.239537 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:53.239567 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:53.239736 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:53.239743 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.241098 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.241101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:53.242809 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:53.242821 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.244014 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.244018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:53.244138 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.244179 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 14:42:53.244309 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 14:42:53.244436 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":121,"thread_start_us":112,"threads_started":1}
I20260808 14:42:53.245139 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.239719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.19.144.65:38961)
0808 14:42:53.240022 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:53.240028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:53.240051 (+ 23us) server_negotiation.cc:407] Connection header received
0808 14:42:53.240155 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:53.240160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:53.240377 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:53.240536 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:53.240806 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.241770 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.243233 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.243924 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.244687 (+ 763us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.244824 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:53.244881 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 14:42:53.244980 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":136,"threads_started":1}
I20260808 14:42:53.245337 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.239286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:42729 (local address 127.0.0.1:42154)
0808 14:42:53.239529 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:53.239542 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:53.239626 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:53.240712 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:53.240715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:53.240747 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:53.240855 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:53.240862 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.242489 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.242492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:53.243591 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:53.243603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.244917 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.244921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:53.245011 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.245034 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 14:42:53.245106 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 14:42:53.245186 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"spinlock_wait_cycles":1280,"thread_start_us":108,"threads_started":1}
I20260808 14:42:53.245764 5636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.239968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42154 (local address 127.19.144.67:42729)
0808 14:42:53.240288 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:53.240299 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:53.240320 (+ 21us) server_negotiation.cc:407] Connection header received
0808 14:42:53.240392 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:53.240396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:53.240570 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:53.240714 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:53.240979 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.242221 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.243737 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.244582 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.244686 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.244824 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:53.245504 (+ 680us) server_negotiation.cc:299] Negotiation successful
0808 14:42:53.245606 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":109,"threads_started":1}
I20260808 14:42:53.248247 5410 tablet_service.cc:1467] Processing CreateTablet for tablet 07f60f5a60204544b0b0d972aad09dc5 (DEFAULT_TABLE table=test-table [id=1cbb2b45a14a492f9bda94262a255315]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:42:53.248394 5272 tablet_service.cc:1467] Processing CreateTablet for tablet 07f60f5a60204544b0b0d972aad09dc5 (DEFAULT_TABLE table=test-table [id=1cbb2b45a14a492f9bda94262a255315]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:42:53.248397 5550 tablet_service.cc:1467] Processing CreateTablet for tablet 07f60f5a60204544b0b0d972aad09dc5 (DEFAULT_TABLE table=test-table [id=1cbb2b45a14a492f9bda94262a255315]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:42:53.249380 5410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07f60f5a60204544b0b0d972aad09dc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:53.249478 5272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07f60f5a60204544b0b0d972aad09dc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:53.249478 5550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07f60f5a60204544b0b0d972aad09dc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:53.256021 5637 tablet_bootstrap.cc:492] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5: Bootstrap starting.
I20260808 14:42:53.257436 5639 tablet_bootstrap.cc:492] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6: Bootstrap starting.
I20260808 14:42:53.258013 5638 tablet_bootstrap.cc:492] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377: Bootstrap starting.
I20260808 14:42:53.258692 5637 tablet_bootstrap.cc:654] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:53.259630 5639 tablet_bootstrap.cc:654] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:53.260113 5637 log.cc:824] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:53.260286 5638 tablet_bootstrap.cc:654] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:53.261170 5638 log.cc:824] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:53.261164 5639 log.cc:824] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:53.262507 5637 tablet_bootstrap.cc:492] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5: No bootstrap required, opened a new log
I20260808 14:42:53.262748 5637 ts_tablet_manager.cc:1397] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 14:42:53.263213 5638 tablet_bootstrap.cc:492] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377: No bootstrap required, opened a new log
I20260808 14:42:53.263219 5639 tablet_bootstrap.cc:492] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6: No bootstrap required, opened a new log
I20260808 14:42:53.263404 5638 ts_tablet_manager.cc:1397] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:42:53.263404 5639 ts_tablet_manager.cc:1397] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:42:53.266104 5637 raft_consensus.cc:348] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55884e0dbe4e488b9e6334afa27d98e5" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36911 } } peers { permanent_uuid: "05dadbdcb7924e68bccb5ae281343377" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 38961 } } peers { permanent_uuid: "1e2c4a92541a4b3eb9ea0378ddbdc6e6" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 42729 } }
I20260808 14:42:53.266373 5637 raft_consensus.cc:374] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:53.266459 5637 raft_consensus.cc:729] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55884e0dbe4e488b9e6334afa27d98e5, State: Initialized, Role: FOLLOWER
I20260808 14:42:53.267234 5637 consensus_queue.cc:260] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55884e0dbe4e488b9e6334afa27d98e5" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36911 } } peers { permanent_uuid: "05dadbdcb7924e68bccb5ae281343377" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 38961 } } peers { permanent_uuid: "1e2c4a92541a4b3eb9ea0378ddbdc6e6" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 42729 } }
I20260808 14:42:53.267442 5639 raft_consensus.cc:348] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55884e0dbe4e488b9e6334afa27d98e5" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36911 } } peers { permanent_uuid: "05dadbdcb7924e68bccb5ae281343377" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 38961 } } peers { permanent_uuid: "1e2c4a92541a4b3eb9ea0378ddbdc6e6" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 42729 } }
I20260808 14:42:53.267781 5639 raft_consensus.cc:374] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:53.267877 5639 raft_consensus.cc:729] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e2c4a92541a4b3eb9ea0378ddbdc6e6, State: Initialized, Role: FOLLOWER
I20260808 14:42:53.267765 5638 raft_consensus.cc:348] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55884e0dbe4e488b9e6334afa27d98e5" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36911 } } peers { permanent_uuid: "05dadbdcb7924e68bccb5ae281343377" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 38961 } } peers { permanent_uuid: "1e2c4a92541a4b3eb9ea0378ddbdc6e6" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 42729 } }
I20260808 14:42:53.268149 5638 raft_consensus.cc:374] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:53.268246 5638 raft_consensus.cc:729] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05dadbdcb7924e68bccb5ae281343377, State: Initialized, Role: FOLLOWER
I20260808 14:42:53.268343 5478 heartbeater.cc:503] Master 127.19.144.126:46463 was elected leader, sending a full tablet report...
I20260808 14:42:53.268481 5637 ts_tablet_manager.cc:1428] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:42:53.268440 5639 consensus_queue.cc:260] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55884e0dbe4e488b9e6334afa27d98e5" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36911 } } peers { permanent_uuid: "05dadbdcb7924e68bccb5ae281343377" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 38961 } } peers { permanent_uuid: "1e2c4a92541a4b3eb9ea0378ddbdc6e6" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 42729 } }
I20260808 14:42:53.268699 5638 consensus_queue.cc:260] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55884e0dbe4e488b9e6334afa27d98e5" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36911 } } peers { permanent_uuid: "05dadbdcb7924e68bccb5ae281343377" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 38961 } } peers { permanent_uuid: "1e2c4a92541a4b3eb9ea0378ddbdc6e6" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 42729 } }
I20260808 14:42:53.269135 5620 heartbeater.cc:503] Master 127.19.144.126:46463 was elected leader, sending a full tablet report...
I20260808 14:42:53.269415 5639 ts_tablet_manager.cc:1428] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 14:42:53.269694 5638 ts_tablet_manager.cc:1428] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260808 14:42:53.269989 5339 heartbeater.cc:503] Master 127.19.144.126:46463 was elected leader, sending a full tablet report...
I20260808 14:42:53.285988 5645 raft_consensus.cc:482] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:42:53.286224 5645 raft_consensus.cc:504] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55884e0dbe4e488b9e6334afa27d98e5" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36911 } } peers { permanent_uuid: "05dadbdcb7924e68bccb5ae281343377" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 38961 } } peers { permanent_uuid: "1e2c4a92541a4b3eb9ea0378ddbdc6e6" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 42729 } }
I20260808 14:42:53.287549 5645 leader_election.cc:288] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55884e0dbe4e488b9e6334afa27d98e5 (127.19.144.66:36911), 1e2c4a92541a4b3eb9ea0378ddbdc6e6 (127.19.144.67:42729)
I20260808 14:42:53.291667 5646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.287851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:36911 (local address 127.19.144.65:49493)
0808 14:42:53.288190 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:53.288207 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:53.288323 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:53.288757 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:53.288759 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:53.288776 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:53.288888 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:53.288893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.289654 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.289660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:53.290487 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:53.290498 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.291190 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.291194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:53.291300 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.291326 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 14:42:53.291427 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 14:42:53.291501 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":153,"threads_started":1}
I20260808 14:42:53.291659 5632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.287914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:49493 (local address 127.19.144.66:36911)
0808 14:42:53.288057 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:53.288061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:53.288291 (+ 230us) server_negotiation.cc:407] Connection header received
0808 14:42:53.288495 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:53.288499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:53.288568 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:53.288612 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:53.289002 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.289515 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.290630 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.291057 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.291195 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.291254 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:53.291447 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 14:42:53.291524 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:42:53.292114 5636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.288486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:48333 (local address 127.19.144.67:42729)
0808 14:42:53.288603 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:53.288606 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:53.288814 (+ 208us) server_negotiation.cc:407] Connection header received
0808 14:42:53.288996 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:53.288999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:53.289054 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:53.289136 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:53.289414 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.290007 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.291021 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.291421 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.291559 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.291608 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:53.291915 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 14:42:53.291995 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 14:42:53.292176 5647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.288310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:42729 (local address 127.19.144.65:48333)
0808 14:42:53.288717 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:53.288728 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:53.288857 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:53.289156 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:53.289159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:53.289179 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:53.289303 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:53.289308 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.290127 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.290130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:53.290886 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:53.290894 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.291552 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.291556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:53.291653 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.291681 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 14:42:53.291772 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 14:42:53.292034 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":1,"thread_start_us":109,"threads_started":1}
I20260808 14:42:53.292385 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f60f5a60204544b0b0d972aad09dc5" candidate_uuid: "05dadbdcb7924e68bccb5ae281343377" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55884e0dbe4e488b9e6334afa27d98e5" is_pre_election: true
I20260808 14:42:53.292752 5430 raft_consensus.cc:2454] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05dadbdcb7924e68bccb5ae281343377 in term 0.
I20260808 14:42:53.292788 5570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f60f5a60204544b0b0d972aad09dc5" candidate_uuid: "05dadbdcb7924e68bccb5ae281343377" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e2c4a92541a4b3eb9ea0378ddbdc6e6" is_pre_election: true
I20260808 14:42:53.293102 5570 raft_consensus.cc:2454] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05dadbdcb7924e68bccb5ae281343377 in term 0.
I20260808 14:42:53.293341 5226 leader_election.cc:302] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05dadbdcb7924e68bccb5ae281343377, 55884e0dbe4e488b9e6334afa27d98e5; no voters:
I20260808 14:42:53.293699 5645 raft_consensus.cc:2781] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:42:53.293780 5645 raft_consensus.cc:482] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:42:53.293830 5645 raft_consensus.cc:3037] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:53.294919 5645 raft_consensus.cc:504] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55884e0dbe4e488b9e6334afa27d98e5" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36911 } } peers { permanent_uuid: "05dadbdcb7924e68bccb5ae281343377" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 38961 } } peers { permanent_uuid: "1e2c4a92541a4b3eb9ea0378ddbdc6e6" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 42729 } }
I20260808 14:42:53.295279 5645 leader_election.cc:288] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [CANDIDATE]: Term 1 election: Requested vote from peers 55884e0dbe4e488b9e6334afa27d98e5 (127.19.144.66:36911), 1e2c4a92541a4b3eb9ea0378ddbdc6e6 (127.19.144.67:42729)
I20260808 14:42:53.295712 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f60f5a60204544b0b0d972aad09dc5" candidate_uuid: "05dadbdcb7924e68bccb5ae281343377" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55884e0dbe4e488b9e6334afa27d98e5"
I20260808 14:42:53.295751 5570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f60f5a60204544b0b0d972aad09dc5" candidate_uuid: "05dadbdcb7924e68bccb5ae281343377" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e2c4a92541a4b3eb9ea0378ddbdc6e6"
I20260808 14:42:53.295874 5430 raft_consensus.cc:3037] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:53.295874 5570 raft_consensus.cc:3037] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:53.296988 5430 raft_consensus.cc:2454] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05dadbdcb7924e68bccb5ae281343377 in term 1.
I20260808 14:42:53.296993 5570 raft_consensus.cc:2454] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05dadbdcb7924e68bccb5ae281343377 in term 1.
I20260808 14:42:53.297380 5226 leader_election.cc:302] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05dadbdcb7924e68bccb5ae281343377, 1e2c4a92541a4b3eb9ea0378ddbdc6e6; no voters:
I20260808 14:42:53.297566 5645 raft_consensus.cc:2781] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:42:53.297883 5645 raft_consensus.cc:686] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [term 1 LEADER]: Becoming Leader. State: Replica: 05dadbdcb7924e68bccb5ae281343377, State: Running, Role: LEADER
I20260808 14:42:53.298287 5645 consensus_queue.cc:237] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55884e0dbe4e488b9e6334afa27d98e5" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36911 } } peers { permanent_uuid: "05dadbdcb7924e68bccb5ae281343377" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 38961 } } peers { permanent_uuid: "1e2c4a92541a4b3eb9ea0378ddbdc6e6" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 42729 } }
I20260808 14:42:53.302003 5145 catalog_manager.cc:5495] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05dadbdcb7924e68bccb5ae281343377 (127.19.144.65). New cstate: current_term: 1 leader_uuid: "05dadbdcb7924e68bccb5ae281343377" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55884e0dbe4e488b9e6334afa27d98e5" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05dadbdcb7924e68bccb5ae281343377" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 38961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e2c4a92541a4b3eb9ea0378ddbdc6e6" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 42729 } health_report { overall_health: UNKNOWN } } }
I20260808 14:42:53.324199 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.320704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41702 (local address 127.19.144.65:38961)
0808 14:42:53.320832 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:53.320843 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:53.320857 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:42:53.320977 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:53.320980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:53.321036 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:53.321127 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:42:53.321400 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.322023 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.322862 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.323075 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.323152 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.323425 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:42:53.323873 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:42:53.323984 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:53.324034 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 14:42:53.324086 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:42:53.330660 5570 raft_consensus.cc:1261] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6 [term 1 FOLLOWER]: Refusing update from remote peer 05dadbdcb7924e68bccb5ae281343377: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:42:53.330660 5430 raft_consensus.cc:1261] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5 [term 1 FOLLOWER]: Refusing update from remote peer 05dadbdcb7924e68bccb5ae281343377: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:42:53.331483 5648 consensus_queue.cc:1035] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e2c4a92541a4b3eb9ea0378ddbdc6e6" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 42729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:42:53.331683 5645 consensus_queue.cc:1035] T 07f60f5a60204544b0b0d972aad09dc5 P 05dadbdcb7924e68bccb5ae281343377 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55884e0dbe4e488b9e6334afa27d98e5" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 36911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:42:53.340875 5653 mvcc.cc:205] Tried to move back new op lower bound from 7316275909954342912 to 7316275909833142272. Current Snapshot: MvccSnapshot[applied={T|T < 7316275909954342912}]
I20260808 14:42:53.348739 5145 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:42:53.348943 5145 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:42:53.351497 5145 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b3614509c3d4900b8fdb1fac7c1ea6b: Sending DeleteTablet for 3 replicas of tablet 07f60f5a60204544b0b0d972aad09dc5
I20260808 14:42:53.352499 5272 tablet_service.cc:1514] Processing DeleteTablet for tablet 07f60f5a60204544b0b0d972aad09dc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:42:53 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:41700
I20260808 14:42:53.352499 5550 tablet_service.cc:1514] Processing DeleteTablet for tablet 07f60f5a60204544b0b0d972aad09dc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:42:53 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:42154
I20260808 14:42:53.352581 5410 tablet_service.cc:1514] Processing DeleteTablet for tablet 07f60f5a60204544b0b0d972aad09dc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:42:53 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:38424
I20260808 14:42:53.353183 5663 tablet_replica.cc:330] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6: stopping tablet replica
I20260808 14:42:53.353264 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 5207
I20260808 14:42:53.353569 5663 raft_consensus.cc:2229] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:42:53.353940 5663 raft_consensus.cc:2258] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:42:53.354405 5664 tablet_replica.cc:330] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5: stopping tablet replica
I20260808 14:42:53.354797 5664 raft_consensus.cc:2229] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:42:53.355147 5664 raft_consensus.cc:2258] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:42:53.356957 5664 ts_tablet_manager.cc:1905] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:42:53.357200 5663 ts_tablet_manager.cc:1905] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:42:53.359655 5663 ts_tablet_manager.cc:1918] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:42:53.359781 5663 log.cc:1196] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/07f60f5a60204544b0b0d972aad09dc5
I20260808 14:42:53.360116 5663 ts_tablet_manager.cc:1939] T 07f60f5a60204544b0b0d972aad09dc5 P 1e2c4a92541a4b3eb9ea0378ddbdc6e6: Deleting consensus metadata
I20260808 14:42:53.360647 5664 ts_tablet_manager.cc:1918] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:42:53.360764 5664 log.cc:1196] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/07f60f5a60204544b0b0d972aad09dc5
I20260808 14:42:53.361146 5664 ts_tablet_manager.cc:1939] T 07f60f5a60204544b0b0d972aad09dc5 P 55884e0dbe4e488b9e6334afa27d98e5: Deleting consensus metadata
I20260808 14:42:53.361440 5131 catalog_manager.cc:4841] TS 1e2c4a92541a4b3eb9ea0378ddbdc6e6 (127.19.144.67:42729): tablet 07f60f5a60204544b0b0d972aad09dc5 (table test-table [id=1cbb2b45a14a492f9bda94262a255315]) successfully deleted
I20260808 14:42:53.362370 5131 catalog_manager.cc:4841] TS 55884e0dbe4e488b9e6334afa27d98e5 (127.19.144.66:36911): tablet 07f60f5a60204544b0b0d972aad09dc5 (table test-table [id=1cbb2b45a14a492f9bda94262a255315]) successfully deleted
W20260808 14:42:53.363487 5131 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.144.65:38961 (error 108)
I20260808 14:42:53.363682 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 5344
I20260808 14:42:53.367583 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.367170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:38961 (local address 127.0.0.1:41712)
0808 14:42:53.367357 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:53.367466 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:38961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 14:42:53.367784 5131 catalog_manager.cc:4584] TS 05dadbdcb7924e68bccb5ae281343377 (127.19.144.65:38961): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 07f60f5a60204544b0b0d972aad09dc5: Network error: Client connection negotiation failed: client connection to 127.19.144.65:38961: connect: Connection refused (error 111)
I20260808 14:42:53.371594 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 5484
I20260808 14:42:53.379573 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 5110
2026-08-08T14:42:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2383 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T14:42:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:42:53Z Disabled control of system clock
I20260808 14:42:53.414463 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:35739
--webserver_interface=127.19.144.126
--webserver_port=0
--builtin_ntp_servers=127.19.144.84:34235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:35739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 14:42:53.535295 5669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:53.535629 5669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:53.535667 5669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:53.539917 5669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:42:53.539996 5669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:53.540014 5669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:53.540031 5669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:42:53.540091 5669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:42:53.545248 5669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:35739
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:35739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:53.546557 5669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:53.548121 5669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:53.557487 5674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:53.557698 5669 server_base.cc:1034] running on GCE node
W20260808 14:42:53.557476 5675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:53.557476 5677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:53.558511 5669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:53.559713 5669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:53.560902 5669 hybrid_clock.cc:648] HybridClock initialized: now 1786200173560869 us; error 49 us; skew 500 ppm
I20260808 14:42:53.562896 5669 webserver.cc:460] Webserver started at http://127.19.144.126:34391/ using document root <none> and password file <none>
I20260808 14:42:53.563473 5669 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:53.563535 5669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:53.563710 5669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:53.566018 5669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "2a18220eeebe4317b58ef3d3896afeba"
format_stamp: "Formatted at 2026-08-08 14:42:53 on dist-test-slave-p3t5"
server_key: "0f9724b97183808a8dad69bce24a085b"
server_key_iv: "2d1ab55e85b875c5e7767217f8801de0"
server_key_version: "encryptionkey@0"
I20260808 14:42:53.566563 5669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "2a18220eeebe4317b58ef3d3896afeba"
format_stamp: "Formatted at 2026-08-08 14:42:53 on dist-test-slave-p3t5"
server_key: "0f9724b97183808a8dad69bce24a085b"
server_key_iv: "2d1ab55e85b875c5e7767217f8801de0"
server_key_version: "encryptionkey@0"
I20260808 14:42:53.570526 5669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:42:53.573052 5683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:53.574415 5669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 14:42:53.574524 5669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "2a18220eeebe4317b58ef3d3896afeba"
format_stamp: "Formatted at 2026-08-08 14:42:53 on dist-test-slave-p3t5"
server_key: "0f9724b97183808a8dad69bce24a085b"
server_key_iv: "2d1ab55e85b875c5e7767217f8801de0"
server_key_version: "encryptionkey@0"
I20260808 14:42:53.574622 5669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:53.600009 5669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:53.600788 5669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:53.600966 5669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:53.608738 5669 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:35739
I20260808 14:42:53.608757 5737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:35739 every 8 connection(s)
I20260808 14:42:53.609851 5669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:42:53.611964 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 5669
I20260808 14:42:53.612084 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:42:53.612423 20033 external_mini_cluster.cc:1428] Setting key 25bd0e935ba9aaa0a7874396c8602271
I20260808 14:42:53.613667 5738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:53.620733 5738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba: Bootstrap starting.
I20260808 14:42:53.622826 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.614082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59498 (local address 127.19.144.126:35739)
0808 14:42:53.614789 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:53.614806 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:53.614844 (+ 38us) server_negotiation.cc:407] Connection header received
0808 14:42:53.615540 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:53.615558 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:53.615999 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:53.616322 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:53.616749 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.617872 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.619129 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.619638 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.619765 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.620791 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:53.620796 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:53.620810 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:42:53.621311 (+ 501us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:53.621370 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:42:53.621402 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:53.621482 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:53.621724 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 14:42:53.622041 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":151,"threads_started":1}
I20260808 14:42:53.623389 5738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:53.624223 5738 log.cc:824] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:53.626543 5738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba: No bootstrap required, opened a new log
I20260808 14:42:53.630481 5738 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a18220eeebe4317b58ef3d3896afeba" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35739 } }
I20260808 14:42:53.630759 5738 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:53.630828 5738 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a18220eeebe4317b58ef3d3896afeba, State: Initialized, Role: FOLLOWER
I20260808 14:42:53.631313 5738 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a18220eeebe4317b58ef3d3896afeba" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35739 } }
I20260808 14:42:53.631456 5738 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:42:53.631505 5738 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:42:53.631582 5738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:53.632493 5738 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a18220eeebe4317b58ef3d3896afeba" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35739 } }
I20260808 14:42:53.632807 5738 leader_election.cc:302] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a18220eeebe4317b58ef3d3896afeba; no voters:
I20260808 14:42:53.633093 5738 leader_election.cc:288] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:42:53.633351 5743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:42:53.633620 5743 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [term 1 LEADER]: Becoming Leader. State: Replica: 2a18220eeebe4317b58ef3d3896afeba, State: Running, Role: LEADER
I20260808 14:42:53.633994 5743 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a18220eeebe4317b58ef3d3896afeba" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35739 } }
I20260808 14:42:53.634483 5738 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:42:53.635743 5745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a18220eeebe4317b58ef3d3896afeba. Latest consensus state: current_term: 1 leader_uuid: "2a18220eeebe4317b58ef3d3896afeba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a18220eeebe4317b58ef3d3896afeba" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35739 } } }
I20260808 14:42:53.635890 5745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [sys.catalog]: This master's current role is: LEADER
I20260808 14:42:53.636238 5744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a18220eeebe4317b58ef3d3896afeba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a18220eeebe4317b58ef3d3896afeba" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 35739 } } }
I20260808 14:42:53.636336 5744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba [sys.catalog]: This master's current role is: LEADER
I20260808 14:42:53.636439 5752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:42:53.639766 5752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:42:53.645598 5752 catalog_manager.cc:1347] Generated new cluster ID: abd6f2d088244226837be2997189c6de
I20260808 14:42:53.645738 5752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:42:53.658363 5752 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:42:53.659387 5752 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:42:53.665807 5752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba: Generated new TSK 0
I20260808 14:42:53.666682 5752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:42:53.669690 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35739
--builtin_ntp_servers=127.19.144.84:34235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 14:42:53.788345 5762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:53.788654 5762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:53.788695 5762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:53.792855 5762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:53.792932 5762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:53.792991 5762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:42:53.797886 5762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:53.799068 5762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:53.800587 5762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:42:53.811481 5762 server_base.cc:1034] running on GCE node
W20260808 14:42:53.811424 5768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:53.811441 5770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:53.811424 5767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:53.812060 5762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:53.812780 5762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:53.813982 5762 hybrid_clock.cc:648] HybridClock initialized: now 1786200173813965 us; error 31 us; skew 500 ppm
I20260808 14:42:53.816047 5762 webserver.cc:460] Webserver started at http://127.19.144.65:35985/ using document root <none> and password file <none>
I20260808 14:42:53.816682 5762 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:53.816784 5762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:53.817044 5762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:53.819334 5762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "a92663eaa6584f718507e378cc4b587f"
format_stamp: "Formatted at 2026-08-08 14:42:53 on dist-test-slave-p3t5"
server_key: "6a7f9a9e8bf64a9c2d81b015019c3652"
server_key_iv: "d017535d2549c8890e22da79a350a6de"
server_key_version: "encryptionkey@0"
I20260808 14:42:53.819948 5762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "a92663eaa6584f718507e378cc4b587f"
format_stamp: "Formatted at 2026-08-08 14:42:53 on dist-test-slave-p3t5"
server_key: "6a7f9a9e8bf64a9c2d81b015019c3652"
server_key_iv: "d017535d2549c8890e22da79a350a6de"
server_key_version: "encryptionkey@0"
I20260808 14:42:53.823951 5762 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 14:42:53.826611 5776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:53.827957 5762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:42:53.828107 5762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "a92663eaa6584f718507e378cc4b587f"
format_stamp: "Formatted at 2026-08-08 14:42:53 on dist-test-slave-p3t5"
server_key: "6a7f9a9e8bf64a9c2d81b015019c3652"
server_key_iv: "d017535d2549c8890e22da79a350a6de"
server_key_version: "encryptionkey@0"
I20260808 14:42:53.828235 5762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:53.839294 5762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:53.839996 5762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:53.840207 5762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:53.840811 5762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:53.841868 5762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:42:53.841939 5762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:53.842016 5762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:42:53.842067 5762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:53.855569 5762 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:34549
I20260808 14:42:53.855600 5892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:34549 every 8 connection(s)
I20260808 14:42:53.856910 5762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:42:53.857735 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 5762
I20260808 14:42:53.857874 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:42:53.858173 20033 external_mini_cluster.cc:1428] Setting key 4055b0b4a1dc60b607ab9a3f2bb61c78
I20260808 14:42:53.860838 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35739
--builtin_ntp_servers=127.19.144.84:34235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 14:42:53.871246 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.860992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:49147 (local address 127.19.144.126:35739)
0808 14:42:53.863110 (+ 2118us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:53.863115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:53.863132 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:42:53.864125 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:53.864129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:53.864191 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:53.864284 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:53.865694 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.866365 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.868270 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.868698 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.868838 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.870542 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:53.870546 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:53.870550 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:42:53.870568 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:53.870597 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:42:53.870610 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:53.870700 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:53.870925 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 14:42:53.871026 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2017}
I20260808 14:42:53.872557 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:53.860015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35739 (local address 127.19.144.65:49147)
0808 14:42:53.862222 (+ 2207us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:53.862287 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:53.863885 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:53.864542 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:53.864555 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:53.864682 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:53.865454 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:53.865479 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.866513 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.866516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:53.868080 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:53.868094 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:53.869066 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:53.869074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:53.869333 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:53.870154 (+ 821us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:42:53.870159 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:53.870227 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:42:53.870335 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:53.870724 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:53.870738 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:53.870746 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 14:42:53.870908 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 14:42:53.871313 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1669,"thread_start_us":129,"threads_started":1}
I20260808 14:42:53.874174 5893 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35739
I20260808 14:42:53.874524 5893 heartbeater.cc:464] Registering TS with master...
I20260808 14:42:53.875293 5893 heartbeater.cc:511] Master 127.19.144.126:35739 requested a full tablet report, sending...
I20260808 14:42:53.877234 5701 ts_manager.cc:194] Registered new tserver with Master: a92663eaa6584f718507e378cc4b587f (127.19.144.65:34549)
I20260808 14:42:53.878969 5701 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.144.65:49147
W20260808 14:42:54.010113 5896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:54.010419 5896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:54.010462 5896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:54.014780 5896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:54.014854 5896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:54.014920 5896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:42:54.019979 5896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:54.021103 5896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:54.022578 5896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:54.032891 5901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:54.032905 5904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:54.032967 5896 server_base.cc:1034] running on GCE node
W20260808 14:42:54.032998 5902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:54.033732 5896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:54.034513 5896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:54.035714 5896 hybrid_clock.cc:648] HybridClock initialized: now 1786200174035674 us; error 53 us; skew 500 ppm
I20260808 14:42:54.037817 5896 webserver.cc:460] Webserver started at http://127.19.144.66:40033/ using document root <none> and password file <none>
I20260808 14:42:54.038452 5896 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:54.038553 5896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:54.038805 5896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:54.041132 5896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "42403840a1e540749c653e75611aa41f"
format_stamp: "Formatted at 2026-08-08 14:42:54 on dist-test-slave-p3t5"
server_key: "96131d09634fae9b7418239f7a1e300f"
server_key_iv: "98438e2652f9fd75a93eaec0dc18eb69"
server_key_version: "encryptionkey@0"
I20260808 14:42:54.041816 5896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "42403840a1e540749c653e75611aa41f"
format_stamp: "Formatted at 2026-08-08 14:42:54 on dist-test-slave-p3t5"
server_key: "96131d09634fae9b7418239f7a1e300f"
server_key_iv: "98438e2652f9fd75a93eaec0dc18eb69"
server_key_version: "encryptionkey@0"
I20260808 14:42:54.045905 5896 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:42:54.048539 5910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:54.049998 5896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:42:54.050148 5896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "42403840a1e540749c653e75611aa41f"
format_stamp: "Formatted at 2026-08-08 14:42:54 on dist-test-slave-p3t5"
server_key: "96131d09634fae9b7418239f7a1e300f"
server_key_iv: "98438e2652f9fd75a93eaec0dc18eb69"
server_key_version: "encryptionkey@0"
I20260808 14:42:54.050278 5896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:54.094082 5896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:54.094901 5896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:54.095129 5896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:54.095791 5896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:54.096840 5896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:42:54.096925 5896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:54.097007 5896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:42:54.097055 5896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:54.106478 5896 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:41353
I20260808 14:42:54.106555 6025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:41353 every 8 connection(s)
I20260808 14:42:54.107575 5896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:42:54.110948 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 5896
I20260808 14:42:54.111114 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:42:54.111418 20033 external_mini_cluster.cc:1428] Setting key bc393723496584b15e3209b550341a25
I20260808 14:42:54.113607 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35739
--builtin_ntp_servers=127.19.144.84:34235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 14:42:54.118112 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.109601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:35141 (local address 127.19.144.126:35739)
0808 14:42:54.109773 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:54.109777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:54.110877 (+ 1100us) server_negotiation.cc:407] Connection header received
0808 14:42:54.112570 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:54.112573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:54.112632 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:54.112744 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:54.113950 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.114784 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.116044 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.116281 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.116418 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:54.117575 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:54.117578 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:54.117581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:42:54.117597 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:54.117644 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:42:54.117657 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:54.117722 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:54.117899 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 14:42:54.117964 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 14:42:54.119041 6028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.110020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35739 (local address 127.19.144.66:35141)
0808 14:42:54.110717 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:54.110760 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:54.112335 (+ 1575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:54.113027 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:54.113045 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:54.113142 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:54.113748 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:54.113770 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.114941 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.114944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:54.115916 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:54.115926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.116442 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.116446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:54.116625 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:54.117293 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:42:54.117295 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:54.117338 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:42:54.117411 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:54.117734 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:54.117747 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:54.117758 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 14:42:54.117894 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 14:42:54.118199 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":123,"threads_started":1}
I20260808 14:42:54.120296 6026 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35739
I20260808 14:42:54.120592 6026 heartbeater.cc:464] Registering TS with master...
I20260808 14:42:54.121352 6026 heartbeater.cc:511] Master 127.19.144.126:35739 requested a full tablet report, sending...
I20260808 14:42:54.122750 5701 ts_manager.cc:194] Registered new tserver with Master: 42403840a1e540749c653e75611aa41f (127.19.144.66:41353)
I20260808 14:42:54.123397 5701 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.144.66:35141
W20260808 14:42:54.239487 6029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:54.239773 6029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:54.239840 6029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:54.244055 6029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:54.244127 6029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:54.244184 6029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:42:54.249225 6029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:35739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:54.250736 6029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:54.252307 6029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:54.262583 6038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:54.262566 6035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:54.262566 6036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:54.262987 6029 server_base.cc:1034] running on GCE node
I20260808 14:42:54.263417 6029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:54.264122 6029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:54.265333 6029 hybrid_clock.cc:648] HybridClock initialized: now 1786200174265313 us; error 33 us; skew 500 ppm
I20260808 14:42:54.267592 6029 webserver.cc:460] Webserver started at http://127.19.144.67:42343/ using document root <none> and password file <none>
I20260808 14:42:54.268250 6029 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:54.268321 6029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:54.268571 6029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:54.270936 6029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "16e955a0a8d84558981e27c136145729"
format_stamp: "Formatted at 2026-08-08 14:42:54 on dist-test-slave-p3t5"
server_key: "005df70c35ad982aec445f305efefb42"
server_key_iv: "1e6c337bb01fbafa25c55e9eb5ed96db"
server_key_version: "encryptionkey@0"
I20260808 14:42:54.271588 6029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "16e955a0a8d84558981e27c136145729"
format_stamp: "Formatted at 2026-08-08 14:42:54 on dist-test-slave-p3t5"
server_key: "005df70c35ad982aec445f305efefb42"
server_key_iv: "1e6c337bb01fbafa25c55e9eb5ed96db"
server_key_version: "encryptionkey@0"
I20260808 14:42:54.275707 6029 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 14:42:54.278523 6044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:54.279976 6029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 14:42:54.280117 6029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "16e955a0a8d84558981e27c136145729"
format_stamp: "Formatted at 2026-08-08 14:42:54 on dist-test-slave-p3t5"
server_key: "005df70c35ad982aec445f305efefb42"
server_key_iv: "1e6c337bb01fbafa25c55e9eb5ed96db"
server_key_version: "encryptionkey@0"
I20260808 14:42:54.280249 6029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:54.298995 6029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:54.299794 6029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:54.300019 6029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:54.300683 6029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:54.301792 6029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:42:54.301865 6029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:54.301911 6029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:42:54.301962 6029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:54.311565 6029 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:35647
I20260808 14:42:54.311595 6161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:35647 every 8 connection(s)
I20260808 14:42:54.312599 6029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:42:54.322265 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 6029
I20260808 14:42:54.322408 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:42:54.322525 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.314777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:44117 (local address 127.19.144.126:35739)
0808 14:42:54.314936 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:54.314953 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:54.316080 (+ 1127us) server_negotiation.cc:407] Connection header received
0808 14:42:54.317490 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:54.317492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:54.317545 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:54.317679 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:54.318647 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.319190 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.320347 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.320563 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.320646 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:54.321976 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:54.321980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:54.321982 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:42:54.321999 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:54.322022 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:42:54.322031 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:54.322117 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:54.322296 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 14:42:54.322369 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 14:42:54.322830 20033 external_mini_cluster.cc:1428] Setting key 2a77dd261f87b200c66e751a74d4d168
I20260808 14:42:54.323696 6164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.315214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:35739 (local address 127.19.144.67:44117)
0808 14:42:54.315905 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:54.315964 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:54.317284 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:54.317862 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:54.317870 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:54.317975 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:54.318499 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:54.318512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.319309 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.319312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:54.320237 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:54.320247 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.320713 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.320717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:54.320873 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:54.321521 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:42:54.321523 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:54.321570 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:42:54.321743 (+ 173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:54.322124 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:54.322136 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:54.322143 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 14:42:54.322295 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 14:42:54.322636 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":110,"threads_started":1}
I20260808 14:42:54.325131 6162 heartbeater.cc:347] Connected to a master server at 127.19.144.126:35739
I20260808 14:42:54.325435 6162 heartbeater.cc:464] Registering TS with master...
I20260808 14:42:54.326079 6162 heartbeater.cc:511] Master 127.19.144.126:35739 requested a full tablet report, sending...
I20260808 14:42:54.327191 5701 ts_manager.cc:194] Registered new tserver with Master: 16e955a0a8d84558981e27c136145729 (127.19.144.67:35647)
I20260808 14:42:54.327783 5701 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.144.67:44117
I20260808 14:42:54.337601 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:42:54.343895 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.339748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59504 (local address 127.19.144.126:35739)
0808 14:42:54.339937 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:54.339942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:54.340100 (+ 158us) server_negotiation.cc:407] Connection header received
0808 14:42:54.340266 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:54.340270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:54.340322 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:54.340411 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:54.340713 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.341241 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.342136 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.342368 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.342494 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:54.343408 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:54.343411 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:54.343414 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:42:54.343428 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:54.343450 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:42:54.343458 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:54.343541 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:54.343669 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 14:42:54.343732 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 14:42:54.347112 5701 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59504:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:42:54.350329 5701 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:42:54.366074 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.360096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:34549 (local address 127.0.0.1:50368)
0808 14:42:54.360600 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:54.360662 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:54.360860 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:54.361696 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:54.361702 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:54.361728 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:54.361897 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:54.361907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.363261 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.363264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:54.364374 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:54.364387 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.365438 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.365442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:54.365668 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:54.365732 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 14:42:54.365831 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 14:42:54.365941 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":37,"thread_start_us":86,"threads_started":1}
I20260808 14:42:54.366187 6174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.360236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50368 (local address 127.19.144.65:34549)
0808 14:42:54.361138 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:54.361144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:54.361166 (+ 22us) server_negotiation.cc:407] Connection header received
0808 14:42:54.361253 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:54.361260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:54.361490 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:54.361712 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:54.362042 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.363138 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.364488 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.365301 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.365437 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:54.365695 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:54.365891 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 14:42:54.366005 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":799,"thread_start_us":130,"threads_started":1}
I20260808 14:42:54.367136 6176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.360881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:41353 (local address 127.0.0.1:41700)
0808 14:42:54.361153 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:54.361167 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:54.361263 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:54.362675 (+ 1412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:54.362678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:54.362698 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:54.362800 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:54.362806 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.364244 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.364247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:54.365463 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:54.365475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.366627 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.366631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:54.366805 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:54.366832 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 14:42:54.366924 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 14:42:54.366999 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":29,"thread_start_us":129,"threads_started":1}
I20260808 14:42:54.367223 6177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.361483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.19.144.66:41353)
0808 14:42:54.362162 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:54.362171 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:54.362192 (+ 21us) server_negotiation.cc:407] Connection header received
0808 14:42:54.362254 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:54.362258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:54.362415 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:54.362587 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:54.362932 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.364123 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.365642 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.366491 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.366603 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:54.366830 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:54.366967 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 14:42:54.367062 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":144,"threads_started":1}
I20260808 14:42:54.370038 5825 tablet_service.cc:1467] Processing CreateTablet for tablet 5813919cca93426993af0804e6e97372 (DEFAULT_TABLE table=test-table [id=384bdb897a104973a76836a7fcf16afe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:42:54.370195 5959 tablet_service.cc:1467] Processing CreateTablet for tablet 5813919cca93426993af0804e6e97372 (DEFAULT_TABLE table=test-table [id=384bdb897a104973a76836a7fcf16afe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:42:54.370645 6175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.360484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:35647 (local address 127.0.0.1:40584)
0808 14:42:54.360764 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:54.360778 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:54.360860 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:54.362517 (+ 1657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:54.362522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:54.362545 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:54.362670 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:54.362677 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.364189 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.364196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:54.366520 (+ 2324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:54.366534 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.370207 (+ 3673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.370211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:54.370316 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:54.370340 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:42:54.370408 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 14:42:54.370493 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":203,"thread_start_us":115,"threads_started":1}
I20260808 14:42:54.370711 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.361510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40584 (local address 127.19.144.67:35647)
0808 14:42:54.362053 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:54.362064 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:54.362089 (+ 25us) server_negotiation.cc:407] Connection header received
0808 14:42:54.362163 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:54.362167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:54.362358 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:54.362512 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:54.362781 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.364034 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.366820 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.370048 (+ 3228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.370194 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:54.370407 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:54.370461 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 14:42:54.370557 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":180,"threads_started":1}
I20260808 14:42:54.371305 5959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5813919cca93426993af0804e6e97372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:54.371305 5825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5813919cca93426993af0804e6e97372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:54.377074 6094 tablet_service.cc:1467] Processing CreateTablet for tablet 5813919cca93426993af0804e6e97372 (DEFAULT_TABLE table=test-table [id=384bdb897a104973a76836a7fcf16afe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:42:54.378392 6094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5813919cca93426993af0804e6e97372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:54.379598 6179 tablet_bootstrap.cc:492] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f: Bootstrap starting.
I20260808 14:42:54.381680 6180 tablet_bootstrap.cc:492] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f: Bootstrap starting.
I20260808 14:42:54.382282 6179 tablet_bootstrap.cc:654] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:54.383868 6179 log.cc:824] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:54.384235 6180 tablet_bootstrap.cc:654] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:54.385135 6180 log.cc:824] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:54.386183 6179 tablet_bootstrap.cc:492] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f: No bootstrap required, opened a new log
I20260808 14:42:54.386412 6179 ts_tablet_manager.cc:1397] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 14:42:54.387244 6180 tablet_bootstrap.cc:492] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f: No bootstrap required, opened a new log
I20260808 14:42:54.387451 6180 ts_tablet_manager.cc:1397] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:42:54.390223 6183 tablet_bootstrap.cc:492] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729: Bootstrap starting.
I20260808 14:42:54.390715 6179 raft_consensus.cc:348] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92663eaa6584f718507e378cc4b587f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34549 } } peers { permanent_uuid: "42403840a1e540749c653e75611aa41f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41353 } } peers { permanent_uuid: "16e955a0a8d84558981e27c136145729" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35647 } }
I20260808 14:42:54.391058 6179 raft_consensus.cc:374] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:54.391119 6179 raft_consensus.cc:729] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a92663eaa6584f718507e378cc4b587f, State: Initialized, Role: FOLLOWER
I20260808 14:42:54.391158 6180 raft_consensus.cc:348] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92663eaa6584f718507e378cc4b587f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34549 } } peers { permanent_uuid: "42403840a1e540749c653e75611aa41f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41353 } } peers { permanent_uuid: "16e955a0a8d84558981e27c136145729" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35647 } }
I20260808 14:42:54.391470 6180 raft_consensus.cc:374] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:54.391531 6180 raft_consensus.cc:729] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42403840a1e540749c653e75611aa41f, State: Initialized, Role: FOLLOWER
I20260808 14:42:54.391635 6179 consensus_queue.cc:260] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92663eaa6584f718507e378cc4b587f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34549 } } peers { permanent_uuid: "42403840a1e540749c653e75611aa41f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41353 } } peers { permanent_uuid: "16e955a0a8d84558981e27c136145729" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35647 } }
I20260808 14:42:54.391956 6180 consensus_queue.cc:260] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92663eaa6584f718507e378cc4b587f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34549 } } peers { permanent_uuid: "42403840a1e540749c653e75611aa41f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41353 } } peers { permanent_uuid: "16e955a0a8d84558981e27c136145729" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35647 } }
I20260808 14:42:54.392457 5893 heartbeater.cc:503] Master 127.19.144.126:35739 was elected leader, sending a full tablet report...
I20260808 14:42:54.392902 6183 tablet_bootstrap.cc:654] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:54.393018 6180 ts_tablet_manager.cc:1428] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:42:54.393417 6026 heartbeater.cc:503] Master 127.19.144.126:35739 was elected leader, sending a full tablet report...
I20260808 14:42:54.394140 6183 log.cc:824] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:54.394795 6179 ts_tablet_manager.cc:1428] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260808 14:42:54.396824 6183 tablet_bootstrap.cc:492] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729: No bootstrap required, opened a new log
I20260808 14:42:54.397104 6183 ts_tablet_manager.cc:1397] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 14:42:54.401427 6183 raft_consensus.cc:348] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92663eaa6584f718507e378cc4b587f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34549 } } peers { permanent_uuid: "42403840a1e540749c653e75611aa41f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41353 } } peers { permanent_uuid: "16e955a0a8d84558981e27c136145729" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35647 } }
I20260808 14:42:54.401819 6183 raft_consensus.cc:374] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:54.401925 6183 raft_consensus.cc:729] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16e955a0a8d84558981e27c136145729, State: Initialized, Role: FOLLOWER
I20260808 14:42:54.402408 6183 consensus_queue.cc:260] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92663eaa6584f718507e378cc4b587f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34549 } } peers { permanent_uuid: "42403840a1e540749c653e75611aa41f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41353 } } peers { permanent_uuid: "16e955a0a8d84558981e27c136145729" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35647 } }
I20260808 14:42:54.403214 6162 heartbeater.cc:503] Master 127.19.144.126:35739 was elected leader, sending a full tablet report...
I20260808 14:42:54.403627 6183 ts_tablet_manager.cc:1428] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260808 14:42:54.501332 6184 raft_consensus.cc:482] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:42:54.501585 6184 raft_consensus.cc:504] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92663eaa6584f718507e378cc4b587f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34549 } } peers { permanent_uuid: "42403840a1e540749c653e75611aa41f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41353 } } peers { permanent_uuid: "16e955a0a8d84558981e27c136145729" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35647 } }
I20260808 14:42:54.502898 6184 leader_election.cc:288] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42403840a1e540749c653e75611aa41f (127.19.144.66:41353), 16e955a0a8d84558981e27c136145729 (127.19.144.67:35647)
I20260808 14:42:54.507069 6177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.503256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:50085 (local address 127.19.144.66:41353)
0808 14:42:54.503401 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:54.503407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:54.503691 (+ 284us) server_negotiation.cc:407] Connection header received
0808 14:42:54.503869 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:54.503874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:54.503930 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:54.504029 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:54.504349 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.504859 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.506081 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.506476 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.506611 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:54.506664 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:54.506851 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 14:42:54.506922 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 14:42:54.507095 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.503379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:59883 (local address 127.19.144.67:35647)
0808 14:42:54.503516 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:54.503521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:54.503715 (+ 194us) server_negotiation.cc:407] Connection header received
0808 14:42:54.503885 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:54.503892 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:54.503937 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:54.504027 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:54.504376 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.504862 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.506095 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.506498 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.506633 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:54.506682 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:54.506895 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 14:42:54.506967 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:42:54.507140 6188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.503174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:35647 (local address 127.19.144.65:59883)
0808 14:42:54.503604 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:54.503616 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:54.503762 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:54.504045 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:54.504048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:54.504092 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:54.504253 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:54.504259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.504990 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.504994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:54.505955 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:54.505966 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.506627 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.506631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:54.506756 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:54.506780 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:42:54.506878 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 14:42:54.506949 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":102,"spinlock_wait_cycles":39296,"thread_start_us":119,"threads_started":1}
I20260808 14:42:54.507331 6189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.503169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:41353 (local address 127.19.144.65:50085)
0808 14:42:54.503577 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:54.503591 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:54.503742 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:54.504042 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:54.504045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:54.504062 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:54.504229 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:54.504234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.504987 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.504990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:54.505948 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:54.505958 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.506604 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.506608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:54.506703 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:54.506741 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 14:42:54.506831 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 14:42:54.506905 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":8,"spinlock_wait_cycles":15232,"thread_start_us":131,"threads_started":1}
I20260808 14:42:54.507997 5980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5813919cca93426993af0804e6e97372" candidate_uuid: "a92663eaa6584f718507e378cc4b587f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42403840a1e540749c653e75611aa41f" is_pre_election: true
I20260808 14:42:54.508047 6114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5813919cca93426993af0804e6e97372" candidate_uuid: "a92663eaa6584f718507e378cc4b587f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16e955a0a8d84558981e27c136145729" is_pre_election: true
I20260808 14:42:54.508322 6114 raft_consensus.cc:2454] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a92663eaa6584f718507e378cc4b587f in term 0.
I20260808 14:42:54.508329 5980 raft_consensus.cc:2454] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a92663eaa6584f718507e378cc4b587f in term 0.
I20260808 14:42:54.508852 5779 leader_election.cc:302] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 16e955a0a8d84558981e27c136145729, 42403840a1e540749c653e75611aa41f, a92663eaa6584f718507e378cc4b587f; no voters:
I20260808 14:42:54.509120 6184 raft_consensus.cc:2781] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:42:54.509212 6184 raft_consensus.cc:482] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:42:54.509264 6184 raft_consensus.cc:3037] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:54.510416 6184 raft_consensus.cc:504] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92663eaa6584f718507e378cc4b587f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34549 } } peers { permanent_uuid: "42403840a1e540749c653e75611aa41f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41353 } } peers { permanent_uuid: "16e955a0a8d84558981e27c136145729" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35647 } }
I20260808 14:42:54.510764 6184 leader_election.cc:288] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [CANDIDATE]: Term 1 election: Requested vote from peers 42403840a1e540749c653e75611aa41f (127.19.144.66:41353), 16e955a0a8d84558981e27c136145729 (127.19.144.67:35647)
I20260808 14:42:54.511121 5980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5813919cca93426993af0804e6e97372" candidate_uuid: "a92663eaa6584f718507e378cc4b587f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42403840a1e540749c653e75611aa41f"
I20260808 14:42:54.511129 6114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5813919cca93426993af0804e6e97372" candidate_uuid: "a92663eaa6584f718507e378cc4b587f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16e955a0a8d84558981e27c136145729"
I20260808 14:42:54.511257 6114 raft_consensus.cc:3037] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:54.511234 5980 raft_consensus.cc:3037] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:54.512440 5980 raft_consensus.cc:2454] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a92663eaa6584f718507e378cc4b587f in term 1.
I20260808 14:42:54.512552 6114 raft_consensus.cc:2454] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a92663eaa6584f718507e378cc4b587f in term 1.
I20260808 14:42:54.512820 5778 leader_election.cc:302] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42403840a1e540749c653e75611aa41f, a92663eaa6584f718507e378cc4b587f; no voters:
I20260808 14:42:54.512993 6184 raft_consensus.cc:2781] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:42:54.513268 6184 raft_consensus.cc:686] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [term 1 LEADER]: Becoming Leader. State: Replica: a92663eaa6584f718507e378cc4b587f, State: Running, Role: LEADER
I20260808 14:42:54.513681 6184 consensus_queue.cc:237] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92663eaa6584f718507e378cc4b587f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34549 } } peers { permanent_uuid: "42403840a1e540749c653e75611aa41f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41353 } } peers { permanent_uuid: "16e955a0a8d84558981e27c136145729" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35647 } }
I20260808 14:42:54.517181 5700 catalog_manager.cc:5495] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f reported cstate change: term changed from 0 to 1, leader changed from <none> to a92663eaa6584f718507e378cc4b587f (127.19.144.65). New cstate: current_term: 1 leader_uuid: "a92663eaa6584f718507e378cc4b587f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a92663eaa6584f718507e378cc4b587f" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42403840a1e540749c653e75611aa41f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16e955a0a8d84558981e27c136145729" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35647 } health_report { overall_health: UNKNOWN } } }
I20260808 14:42:54.560251 6174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.555555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50384 (local address 127.19.144.65:34549)
0808 14:42:54.555683 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:54.555687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:54.555701 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:42:54.555845 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:54.555848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:54.555908 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:54.556022 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:42:54.556306 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.556799 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.558017 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:54.558436 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:54.558557 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:54.559084 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:42:54.559727 (+ 643us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:42:54.559873 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:54.559977 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 14:42:54.560054 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 14:42:54.563683 6163 tablet.cc:2369] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:42:54.568878 5980 raft_consensus.cc:1261] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f [term 1 FOLLOWER]: Refusing update from remote peer a92663eaa6584f718507e378cc4b587f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:42:54.568869 6114 raft_consensus.cc:1261] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729 [term 1 FOLLOWER]: Refusing update from remote peer a92663eaa6584f718507e378cc4b587f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:42:54.569717 6190 consensus_queue.cc:1035] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [LEADER]: Connected to new peer: Peer: permanent_uuid: "16e955a0a8d84558981e27c136145729" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:42:54.569917 6184 consensus_queue.cc:1035] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42403840a1e540749c653e75611aa41f" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 41353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:42:54.579818 6195 mvcc.cc:205] Tried to move back new op lower bound from 7316275915024793600 to 7316275914810896384. Current Snapshot: MvccSnapshot[applied={T|T < 7316275915024793600}]
I20260808 14:42:54.587095 5701 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:42:54.587313 5701 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:42:54.590088 5701 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a18220eeebe4317b58ef3d3896afeba: Sending DeleteTablet for 3 replicas of tablet 5813919cca93426993af0804e6e97372
I20260808 14:42:54.591066 6094 tablet_service.cc:1514] Processing DeleteTablet for tablet 5813919cca93426993af0804e6e97372 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:42:54 UTC) from {username='slave'} at 127.0.0.1:40584
I20260808 14:42:54.591066 5825 tablet_service.cc:1514] Processing DeleteTablet for tablet 5813919cca93426993af0804e6e97372 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:42:54 UTC) from {username='slave'} at 127.0.0.1:50368
I20260808 14:42:54.591096 5959 tablet_service.cc:1514] Processing DeleteTablet for tablet 5813919cca93426993af0804e6e97372 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:42:54 UTC) from {username='slave'} at 127.0.0.1:41700
I20260808 14:42:54.591638 6205 tablet_replica.cc:330] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f: stopping tablet replica
I20260808 14:42:54.591899 6204 tablet_replica.cc:330] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f: stopping tablet replica
I20260808 14:42:54.592000 6205 raft_consensus.cc:2229] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:42:54.592285 6204 raft_consensus.cc:2229] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:42:54.592372 6205 raft_consensus.cc:2258] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:42:54.592471 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 5762
I20260808 14:42:54.592832 6204 raft_consensus.cc:2258] T 5813919cca93426993af0804e6e97372 P a92663eaa6584f718507e378cc4b587f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:42:54.593314 6203 tablet_replica.cc:330] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729: stopping tablet replica
I20260808 14:42:54.593672 6203 raft_consensus.cc:2229] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:42:54.593988 6203 raft_consensus.cc:2258] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:42:54.595849 6203 ts_tablet_manager.cc:1905] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:42:54.598356 6203 ts_tablet_manager.cc:1918] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:42:54.598487 6203 log.cc:1196] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/5813919cca93426993af0804e6e97372
I20260808 14:42:54.598898 6203 ts_tablet_manager.cc:1939] T 5813919cca93426993af0804e6e97372 P 16e955a0a8d84558981e27c136145729: Deleting consensus metadata
I20260808 14:42:54.600023 6205 ts_tablet_manager.cc:1905] T 5813919cca93426993af0804e6e97372 P 42403840a1e540749c653e75611aa41f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:42:54.600412 5686 catalog_manager.cc:4841] TS 16e955a0a8d84558981e27c136145729 (127.19.144.67:35647): tablet 5813919cca93426993af0804e6e97372 (table test-table [id=384bdb897a104973a76836a7fcf16afe]) successfully deleted
W20260808 14:42:54.601380 5685 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.144.65:34549 (error 108)
I20260808 14:42:54.602411 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 5896
I20260808 14:42:54.602633 6175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.602282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:34549 (local address 127.0.0.1:50394)
0808 14:42:54.602431 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:54.602524 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:34549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 14:42:54.604053 5685 catalog_manager.cc:4584] TS a92663eaa6584f718507e378cc4b587f (127.19.144.65:34549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5813919cca93426993af0804e6e97372: Network error: Client connection negotiation failed: client connection to 127.19.144.65:34549: connect: Connection refused (error 111)
I20260808 14:42:54.610730 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 6029
I20260808 14:42:54.613436 6175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:54.610947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:41353 (local address 127.0.0.1:41704)
0808 14:42:54.613245 (+ 2298us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:54.613327 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.66:41353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2170}
I20260808 14:42:54.618822 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 5669
2026-08-08T14:42:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1238 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:42:54 dist-test-slave-p3t5 krb5kdc[6210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:42:54 dist-test-slave-p3t5 krb5kdc[6210](info): set up 2 sockets
Aug 08 14:42:54 dist-test-slave-p3t5 krb5kdc[6210](info): commencing operation
krb5kdc: starting...
W20260808 14:42:55.682993 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:42:55 dist-test-slave-p3t5 krb5kdc[6210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:42:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:42:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:42:55.819396 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:34351
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:34351
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:42:55.938898 6223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:55.939262 6223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:55.939348 6223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:55.943476 6223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:42:55.943583 6223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:55.943624 6223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:55.943672 6223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:42:55.943725 6223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:42:55.950008 6223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:34351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:34351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:55.951272 6223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:55.952431 6223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:55.961650 6229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:55.961712 6228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:55.961671 6231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:55.962016 6223 server_base.cc:1034] running on GCE node
I20260808 14:42:55.962716 6223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:55.963922 6223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:55.965112 6223 hybrid_clock.cc:648] HybridClock initialized: now 1786200175965076 us; error 49 us; skew 500 ppm
Aug 08 14:42:55 dist-test-slave-p3t5 krb5kdc[6210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200175, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:55.968264 6223 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:42:55.969231 6223 webserver.cc:460] Webserver started at http://127.19.144.126:40687/ using document root <none> and password file <none>
I20260808 14:42:55.969882 6223 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:55.969960 6223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:55.970141 6223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:55.972455 6223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "3bbb99efbaf04b789a6fb66f5ef93c9f"
format_stamp: "Formatted at 2026-08-08 14:42:55 on dist-test-slave-p3t5"
server_key: "b6cf1e443c19e72cecb60f0b68fb6efa"
server_key_iv: "f697ea6c64b9962baf0d5bea05d0cea8"
server_key_version: "encryptionkey@0"
I20260808 14:42:55.973018 6223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "3bbb99efbaf04b789a6fb66f5ef93c9f"
format_stamp: "Formatted at 2026-08-08 14:42:55 on dist-test-slave-p3t5"
server_key: "b6cf1e443c19e72cecb60f0b68fb6efa"
server_key_iv: "f697ea6c64b9962baf0d5bea05d0cea8"
server_key_version: "encryptionkey@0"
I20260808 14:42:55.977074 6223 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:42:55.979736 6238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:55.981074 6223 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:42:55.981230 6223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "3bbb99efbaf04b789a6fb66f5ef93c9f"
format_stamp: "Formatted at 2026-08-08 14:42:55 on dist-test-slave-p3t5"
server_key: "b6cf1e443c19e72cecb60f0b68fb6efa"
server_key_iv: "f697ea6c64b9962baf0d5bea05d0cea8"
server_key_version: "encryptionkey@0"
I20260808 14:42:55.981359 6223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:56.020049 6223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:56.023416 6223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:56.023658 6223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:56.031659 6223 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:34351
I20260808 14:42:56.031685 6291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:34351 every 8 connection(s)
I20260808 14:42:56.032747 6223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:42:56.035785 6292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:56.037870 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 6223
I20260808 14:42:56.037987 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:42:56.038290 20033 external_mini_cluster.cc:1428] Setting key 9ce5346e1633cd06c69c252142d144d0
I20260808 14:42:56.042271 6292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f: Bootstrap starting.
I20260808 14:42:56.044638 6292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:56.045418 6292 log.cc:824] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:56.048023 6292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f: No bootstrap required, opened a new log
Aug 08 14:42:56 dist-test-slave-p3t5 krb5kdc[6210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:42:56.053042 6292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bbb99efbaf04b789a6fb66f5ef93c9f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34351 } }
I20260808 14:42:56.053350 6292 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:56.053411 6292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bbb99efbaf04b789a6fb66f5ef93c9f, State: Initialized, Role: FOLLOWER
I20260808 14:42:56.053949 6292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bbb99efbaf04b789a6fb66f5ef93c9f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34351 } }
I20260808 14:42:56.054112 6292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:42:56.054180 6292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:42:56.054277 6292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:56.054441 6295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:56.039839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37512 (local address 127.19.144.126:34351)
0808 14:42:56.040338 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:56.040379 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:56.040421 (+ 42us) server_negotiation.cc:407] Connection header received
0808 14:42:56.041285 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:56.041309 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:56.041800 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:56.042161 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:56.043109 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.044172 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.045104 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.045659 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.045849 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:56.048750 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:56.048757 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:56.048774 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:56.048807 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:56.050972 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:56.051429 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:56.051436 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:56.051442 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:56.051597 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:56.051957 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:56.051960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:56.051962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:56.052547 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:56.052761 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:56.053036 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 14:42:56.053355 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":157,"threads_started":1}
I20260808 14:42:56.055351 6292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bbb99efbaf04b789a6fb66f5ef93c9f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34351 } }
I20260808 14:42:56.055708 6292 leader_election.cc:302] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bbb99efbaf04b789a6fb66f5ef93c9f; no voters:
I20260808 14:42:56.056030 6292 leader_election.cc:288] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:42:56.056144 6297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:42:56.056373 6297 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [term 1 LEADER]: Becoming Leader. State: Replica: 3bbb99efbaf04b789a6fb66f5ef93c9f, State: Running, Role: LEADER
I20260808 14:42:56.056712 6297 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bbb99efbaf04b789a6fb66f5ef93c9f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34351 } }
I20260808 14:42:56.057358 6292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:42:56.058532 6299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bbb99efbaf04b789a6fb66f5ef93c9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bbb99efbaf04b789a6fb66f5ef93c9f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34351 } } }
I20260808 14:42:56.058532 6298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bbb99efbaf04b789a6fb66f5ef93c9f. Latest consensus state: current_term: 1 leader_uuid: "3bbb99efbaf04b789a6fb66f5ef93c9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bbb99efbaf04b789a6fb66f5ef93c9f" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34351 } } }
I20260808 14:42:56.058751 6298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [sys.catalog]: This master's current role is: LEADER
I20260808 14:42:56.058703 6299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f [sys.catalog]: This master's current role is: LEADER
I20260808 14:42:56.059158 6306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:42:56.062495 6306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:42:56.068404 6306 catalog_manager.cc:1347] Generated new cluster ID: e095a4f52efc49e5b4cd40873888178f
I20260808 14:42:56.068495 6306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:42:56.086243 6306 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:42:56.087708 6306 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:42:56.096350 6306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f: Generated new TSK 0
I20260808 14:42:56.097309 6306 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:42:56.136482 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34351
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:42:56.258733 6319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:56.259042 6319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:56.259130 6319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:56.263990 6319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:56.264060 6319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:56.264122 6319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:42:56.269711 6319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:56.270889 6319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:56.272058 6319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:56.282150 6327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:56.282150 6325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:56.282193 6324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:56.282405 6319 server_base.cc:1034] running on GCE node
I20260808 14:42:56.282927 6319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:56.283602 6319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:56.284765 6319 hybrid_clock.cc:648] HybridClock initialized: now 1786200176284743 us; error 53 us; skew 500 ppm
Aug 08 14:42:56 dist-test-slave-p3t5 krb5kdc[6210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200176, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:56.291386 6319 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:42:56.292392 6319 webserver.cc:460] Webserver started at http://127.19.144.65:38829/ using document root <none> and password file <none>
I20260808 14:42:56.293013 6319 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:56.293112 6319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:56.293357 6319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:56.295630 6319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "de8b6f76f89b4f2396f0e11e74361ff1"
format_stamp: "Formatted at 2026-08-08 14:42:56 on dist-test-slave-p3t5"
server_key: "ea8de2c4adc14574e416de98b58583d5"
server_key_iv: "dc780ed23f15668875a40997f220cb2c"
server_key_version: "encryptionkey@0"
I20260808 14:42:56.296247 6319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "de8b6f76f89b4f2396f0e11e74361ff1"
format_stamp: "Formatted at 2026-08-08 14:42:56 on dist-test-slave-p3t5"
server_key: "ea8de2c4adc14574e416de98b58583d5"
server_key_iv: "dc780ed23f15668875a40997f220cb2c"
server_key_version: "encryptionkey@0"
I20260808 14:42:56.300184 6319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 14:42:56.302763 6334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:56.304049 6319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:42:56.304191 6319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "de8b6f76f89b4f2396f0e11e74361ff1"
format_stamp: "Formatted at 2026-08-08 14:42:56 on dist-test-slave-p3t5"
server_key: "ea8de2c4adc14574e416de98b58583d5"
server_key_iv: "dc780ed23f15668875a40997f220cb2c"
server_key_version: "encryptionkey@0"
I20260808 14:42:56.304306 6319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:56.322887 6319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:56.326186 6319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:56.326408 6319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:56.327067 6319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:56.327960 6319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:42:56.328032 6319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:56.328104 6319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:42:56.328151 6319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:56.337196 6319 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:34771
I20260808 14:42:56.337217 6453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:34771 every 8 connection(s)
I20260808 14:42:56.338294 6319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:42:56.345088 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 6319
I20260808 14:42:56.345260 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:42:56.345556 20033 external_mini_cluster.cc:1428] Setting key c0a7c8ee87eb6f5ece3cf4b29fafa9ff
Aug 08 14:42:56 dist-test-slave-p3t5 krb5kdc[6210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200176, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:42:56.353652 6295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:56.340237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:40955 (local address 127.19.144.126:34351)
0808 14:42:56.340383 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:56.340387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:56.341277 (+ 890us) server_negotiation.cc:407] Connection header received
0808 14:42:56.342772 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:56.342776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:56.342840 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:56.342946 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:56.344208 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.344764 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.346131 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.346406 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.346516 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:56.349419 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:56.349423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:56.349425 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:56.349453 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:56.351035 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:56.351723 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:56.351726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:56.351727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:56.351810 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:56.352273 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:56.352275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:56.352277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:56.352545 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:56.352629 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:56.353345 (+ 716us) server_negotiation.cc:299] Negotiation successful
0808 14:42:56.353476 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:42:56.355109 6457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:56.340643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:34351 (local address 127.19.144.65:40955)
0808 14:42:56.341124 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:56.341170 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:56.342537 (+ 1367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:56.343148 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:56.343155 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:56.343537 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:56.344052 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:56.344066 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.344912 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.344916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:56.346002 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:56.346012 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.346581 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.346584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:56.346756 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:56.347477 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:42:56.347494 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:56.349242 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:56.351165 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:56.351174 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:56.351194 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:56.351621 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:56.351904 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:56.351910 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:56.351915 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:56.352163 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:56.352664 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:56.352672 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:56.353063 (+ 391us) client_negotiation.cc:769] Sending connection context
0808 14:42:56.353318 (+ 255us) client_negotiation.cc:240] Negotiation successful
0808 14:42:56.353823 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":122,"threads_started":1}
I20260808 14:42:56.356608 6455 heartbeater.cc:347] Connected to a master server at 127.19.144.126:34351
I20260808 14:42:56.356907 6455 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:42:56.357596 6455 heartbeater.cc:511] Master 127.19.144.126:34351 requested a full tablet report, sending...
I20260808 14:42:56.359694 6255 ts_manager.cc:194] Registered new tserver with Master: de8b6f76f89b4f2396f0e11e74361ff1 (127.19.144.65:34771)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
I20260808 14:42:56.361078 6255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:40955
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:42:56.378371 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34351
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:42:56.500797 6460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:56.501147 6460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:56.501230 6460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:56.505491 6460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:56.505600 6460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:56.505722 6460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:42:56.511379 6460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:56.512578 6460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:56.513741 6460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:56.524062 6465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:56.524189 6460 server_base.cc:1034] running on GCE node
W20260808 14:42:56.524060 6468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:56.524060 6466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:56.524766 6460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:56.525456 6460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:56.526639 6460 hybrid_clock.cc:648] HybridClock initialized: now 1786200176526618 us; error 43 us; skew 500 ppm
Aug 08 14:42:56 dist-test-slave-p3t5 krb5kdc[6210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200176, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:56.529448 6460 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:42:56.530376 6460 webserver.cc:460] Webserver started at http://127.19.144.66:35797/ using document root <none> and password file <none>
I20260808 14:42:56.530972 6460 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:56.531064 6460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:56.531296 6460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:56.533414 6460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "e892fdb075dc4a309b0583f93a41f0b6"
format_stamp: "Formatted at 2026-08-08 14:42:56 on dist-test-slave-p3t5"
server_key: "b19b9ff6f5494cc33186c34f605cc2ff"
server_key_iv: "dc1be1f726419c99c57524055c98f26f"
server_key_version: "encryptionkey@0"
I20260808 14:42:56.534073 6460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "e892fdb075dc4a309b0583f93a41f0b6"
format_stamp: "Formatted at 2026-08-08 14:42:56 on dist-test-slave-p3t5"
server_key: "b19b9ff6f5494cc33186c34f605cc2ff"
server_key_iv: "dc1be1f726419c99c57524055c98f26f"
server_key_version: "encryptionkey@0"
I20260808 14:42:56.538133 6460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:42:56.540784 6475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:56.542151 6460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:42:56.542296 6460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "e892fdb075dc4a309b0583f93a41f0b6"
format_stamp: "Formatted at 2026-08-08 14:42:56 on dist-test-slave-p3t5"
server_key: "b19b9ff6f5494cc33186c34f605cc2ff"
server_key_iv: "dc1be1f726419c99c57524055c98f26f"
server_key_version: "encryptionkey@0"
I20260808 14:42:56.542420 6460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:56.554267 6460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:56.557166 6460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:56.557394 6460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:56.558218 6460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:56.559446 6460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:42:56.559512 6460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:56.559594 6460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:42:56.559667 6460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:56.569998 6460 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:40419
I20260808 14:42:56.570019 6591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:40419 every 8 connection(s)
I20260808 14:42:56.570923 6460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:42:56.576787 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 6460
I20260808 14:42:56.576922 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:42:56.577265 20033 external_mini_cluster.cc:1428] Setting key 9bb1b5dcdf6366e91bace9654a76e8d5
Aug 08 14:42:56 dist-test-slave-p3t5 krb5kdc[6210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200176, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:42:56.586395 6295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:56.572768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:59391 (local address 127.19.144.126:34351)
0808 14:42:56.572903 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:56.572907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:56.573809 (+ 902us) server_negotiation.cc:407] Connection header received
0808 14:42:56.575202 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:56.575206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:56.575262 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:56.575345 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:56.576521 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.577193 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.578453 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.578685 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.578795 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:56.581557 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:56.581561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:56.581563 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:56.581589 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:56.583538 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:56.584122 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:56.584130 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:56.584134 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:56.584245 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:56.584849 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:56.584854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:56.584856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:56.585179 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:56.585315 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:56.585977 (+ 662us) server_negotiation.cc:299] Negotiation successful
0808 14:42:56.586137 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 14:42:56.587285 6594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:56.573165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:34351 (local address 127.19.144.66:59391)
0808 14:42:56.573659 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:56.573714 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:56.574989 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:56.575518 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:56.575525 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:56.575884 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:56.576375 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:56.576390 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.577320 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.577323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:56.578317 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:56.578327 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.578838 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.578842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:56.579012 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:56.579640 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:42:56.579656 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:56.581350 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:56.583679 (+ 2329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:56.583685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:56.583699 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:56.584003 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:56.584445 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:56.584451 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:56.584453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:56.584734 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:56.585320 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:56.585326 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:56.585598 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 14:42:56.585884 (+ 286us) client_negotiation.cc:240] Negotiation successful
0808 14:42:56.586284 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":114,"threads_started":1}
I20260808 14:42:56.588660 6592 heartbeater.cc:347] Connected to a master server at 127.19.144.126:34351
I20260808 14:42:56.588958 6592 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
I20260808 14:42:56.589676 6592 heartbeater.cc:511] Master 127.19.144.126:34351 requested a full tablet report, sending...
I20260808 14:42:56.590910 6255 ts_manager.cc:194] Registered new tserver with Master: e892fdb075dc4a309b0583f93a41f0b6 (127.19.144.66:40419)
I20260808 14:42:56.591543 6255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:59391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:42:56.613754 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34351
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:36013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:42:56.734426 6597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:56.734798 6597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:56.734894 6597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:56.739234 6597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:56.739341 6597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:56.739435 6597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:42:56.745143 6597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:36013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:56.746440 6597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:56.747653 6597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:56.758684 6604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:56.758841 6606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:56.758687 6603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:56.759080 6597 server_base.cc:1034] running on GCE node
I20260808 14:42:56.759580 6597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:56.760351 6597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:56.761575 6597 hybrid_clock.cc:648] HybridClock initialized: now 1786200176761536 us; error 49 us; skew 500 ppm
Aug 08 14:42:56 dist-test-slave-p3t5 krb5kdc[6210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200176, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:56.764568 6597 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:42:56.765527 6597 webserver.cc:460] Webserver started at http://127.19.144.67:36179/ using document root <none> and password file <none>
I20260808 14:42:56.766175 6597 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:56.766270 6597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:56.766520 6597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:56.768733 6597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "3ae4b13576ec4452b41f98d876b6e735"
format_stamp: "Formatted at 2026-08-08 14:42:56 on dist-test-slave-p3t5"
server_key: "1892d4d3fb28c4e853fc08ea6299871e"
server_key_iv: "92ae31ebf288513b11352478a793c080"
server_key_version: "encryptionkey@0"
I20260808 14:42:56.769400 6597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "3ae4b13576ec4452b41f98d876b6e735"
format_stamp: "Formatted at 2026-08-08 14:42:56 on dist-test-slave-p3t5"
server_key: "1892d4d3fb28c4e853fc08ea6299871e"
server_key_iv: "92ae31ebf288513b11352478a793c080"
server_key_version: "encryptionkey@0"
I20260808 14:42:56.773766 6597 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:42:56.776470 6613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:56.777868 6597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:42:56.778016 6597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "3ae4b13576ec4452b41f98d876b6e735"
format_stamp: "Formatted at 2026-08-08 14:42:56 on dist-test-slave-p3t5"
server_key: "1892d4d3fb28c4e853fc08ea6299871e"
server_key_iv: "92ae31ebf288513b11352478a793c080"
server_key_version: "encryptionkey@0"
I20260808 14:42:56.778143 6597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:56.795925 6597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:56.799015 6597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:56.799235 6597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:56.799875 6597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:56.800802 6597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:42:56.800882 6597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:56.800973 6597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:42:56.801021 6597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:56.810631 6597 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:35037
I20260808 14:42:56.810650 6731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:35037 every 8 connection(s)
I20260808 14:42:56.811684 6597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:42:56.812067 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 6597
I20260808 14:42:56.812215 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:42:56.812572 20033 external_mini_cluster.cc:1428] Setting key 32b8fef9d102eec279d622c048b3ad34
Aug 08 14:42:56 dist-test-slave-p3t5 krb5kdc[6210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200176, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:42:56.825711 6295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:56.813742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:36029 (local address 127.19.144.126:34351)
0808 14:42:56.813896 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:56.813901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:56.814902 (+ 1001us) server_negotiation.cc:407] Connection header received
0808 14:42:56.816327 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:56.816330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:56.816377 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:56.816448 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:56.817707 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.818247 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.819363 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.819575 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.819649 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:56.822362 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:56.822365 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:56.822367 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:56.822392 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:56.823712 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:56.824202 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:56.824205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:56.824206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:56.824284 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:56.824596 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:56.824599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:56.824600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:56.824783 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:56.824858 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:56.825377 (+ 519us) server_negotiation.cc:299] Negotiation successful
0808 14:42:56.825511 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:42:56.826735 6734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:56.814240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:34351 (local address 127.19.144.67:36029)
0808 14:42:56.814759 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:56.814816 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:56.816127 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:56.816617 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:56.816623 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:56.817027 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:56.817530 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:56.817545 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.818350 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.818353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:56.819258 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:56.819266 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.819703 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.819705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:56.819848 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:56.820505 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:42:56.820522 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:56.822194 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:56.823817 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:56.823822 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:56.823836 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:56.824108 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:56.824357 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:42:56.824359 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:42:56.824361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:42:56.824507 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:42:56.824866 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:56.824871 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:56.825120 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 14:42:56.825310 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 14:42:56.825717 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":151,"threads_started":1}
I20260808 14:42:56.827868 6732 heartbeater.cc:347] Connected to a master server at 127.19.144.126:34351
I20260808 14:42:56.828130 6732 heartbeater.cc:464] Registering TS with master...
I20260808 14:42:56.828642 6732 heartbeater.cc:511] Master 127.19.144.126:34351 requested a full tablet report, sending...
I20260808 14:42:56.829707 6255 ts_manager.cc:194] Registered new tserver with Master: 3ae4b13576ec4452b41f98d876b6e735 (127.19.144.67:35037)
I20260808 14:42:56.830286 6255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:36029
I20260808 14:42:56.839099 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:42:56.849581 6295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:56.841206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37526 (local address 127.19.144.126:34351)
0808 14:42:56.841362 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:56.841369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:56.841541 (+ 172us) server_negotiation.cc:407] Connection header received
0808 14:42:56.841738 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:56.841742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:56.841794 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:56.841884 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:56.842665 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.843195 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.844062 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.844282 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.844389 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:56.846138 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:56.846153 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:56.846156 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:42:56.846212 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:56.847673 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:56.848213 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:56.848222 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:56.848224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:56.848304 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:42:56.848638 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:42:56.848641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:42:56.848643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:42:56.848865 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:56.849033 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:56.849305 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 14:42:56.849403 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:42:56.852741 6255 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:42:56.855934 6255 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:42:56.871632 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:56.865401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:34771 (local address 127.0.0.1:56846)
0808 14:42:56.866445 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:56.866503 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:56.866655 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:56.867395 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:56.867400 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:56.867419 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:56.867529 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:56.867535 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.868875 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.868881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:56.870124 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:56.870134 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.871053 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.871056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:56.871161 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:56.871199 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 14:42:56.871317 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 14:42:56.871449 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"mutex_wait_us":25,"spinlock_wait_cycles":199552,"thread_start_us":90,"threads_started":1}
I20260808 14:42:56.871814 6747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:56.865719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46772 (local address 127.19.144.67:35037)
0808 14:42:56.866187 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:56.866201 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:56.866585 (+ 384us) server_negotiation.cc:407] Connection header received
0808 14:42:56.866821 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:56.866827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:56.867055 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:56.867204 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:56.867510 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.868861 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.870292 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.871134 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.871305 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:56.871460 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:56.871534 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 14:42:56.871637 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":144,"threads_started":1}
I20260808 14:42:56.872226 6748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:56.865852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56846 (local address 127.19.144.65:34771)
0808 14:42:56.866083 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:56.866090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:56.866604 (+ 514us) server_negotiation.cc:407] Connection header received
0808 14:42:56.866827 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:56.866838 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:56.867054 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:56.867198 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:56.867653 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.868739 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.870249 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.870920 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.871051 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:56.871218 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:56.871951 (+ 733us) server_negotiation.cc:299] Negotiation successful
0808 14:42:56.872060 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":70,"threads_started":1}
I20260808 14:42:56.872326 6745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:56.865137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38528 (local address 127.19.144.66:40419)
0808 14:42:56.866398 (+ 1261us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:56.866408 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:56.866963 (+ 555us) server_negotiation.cc:407] Connection header received
0808 14:42:56.867046 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:56.867051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:56.867209 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:56.867360 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:56.867701 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.868441 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.870441 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.871255 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.871347 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:56.871467 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:56.872044 (+ 577us) server_negotiation.cc:299] Negotiation successful
0808 14:42:56.872163 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1168,"thread_start_us":147,"threads_started":1}
I20260808 14:42:56.871632 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:56.865150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:35037 (local address 127.0.0.1:46772)
0808 14:42:56.866451 (+ 1301us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:56.866504 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:56.866655 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:56.867221 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:56.867227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:56.867250 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:56.867394 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:56.867400 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.869008 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.869011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:56.870151 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:56.870160 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.871297 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.871304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:56.871396 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:56.871417 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 14:42:56.871482 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 14:42:56.871538 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"mutex_wait_us":49,"spinlock_wait_cycles":1093632,"thread_start_us":114,"threads_started":1}
I20260808 14:42:56.871944 6746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:56.864982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:40419 (local address 127.0.0.1:38528)
0808 14:42:56.866841 (+ 1859us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:56.866854 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:56.866944 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:56.867507 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:56.867511 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:56.867524 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:56.867606 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:56.867614 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.868577 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.868580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:56.870329 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:56.870335 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:56.871531 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:56.871534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:56.871634 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:56.871657 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 14:42:56.871728 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 14:42:56.871794 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":42,"thread_start_us":99,"threads_started":1}
I20260808 14:42:56.875846 6386 tablet_service.cc:1467] Processing CreateTablet for tablet 97a95f6aa3be4df1984478d87f7152d6 (DEFAULT_TABLE table=test-table [id=d9b571dc86ab4999acb6abab047248a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:42:56.875846 6525 tablet_service.cc:1467] Processing CreateTablet for tablet 97a95f6aa3be4df1984478d87f7152d6 (DEFAULT_TABLE table=test-table [id=d9b571dc86ab4999acb6abab047248a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:42:56.877092 6386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97a95f6aa3be4df1984478d87f7152d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:56.877094 6525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97a95f6aa3be4df1984478d87f7152d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:56.878533 6664 tablet_service.cc:1467] Processing CreateTablet for tablet 97a95f6aa3be4df1984478d87f7152d6 (DEFAULT_TABLE table=test-table [id=d9b571dc86ab4999acb6abab047248a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:42:56.879560 6664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97a95f6aa3be4df1984478d87f7152d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:56.884925 6749 tablet_bootstrap.cc:492] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1: Bootstrap starting.
I20260808 14:42:56.885457 6750 tablet_bootstrap.cc:492] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6: Bootstrap starting.
I20260808 14:42:56.888247 6750 tablet_bootstrap.cc:654] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:56.888581 6749 tablet_bootstrap.cc:654] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:56.889636 6749 log.cc:824] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:56.889633 6750 log.cc:824] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:56.890093 6752 tablet_bootstrap.cc:492] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735: Bootstrap starting.
I20260808 14:42:56.892109 6750 tablet_bootstrap.cc:492] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6: No bootstrap required, opened a new log
I20260808 14:42:56.892109 6749 tablet_bootstrap.cc:492] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1: No bootstrap required, opened a new log
I20260808 14:42:56.892216 6752 tablet_bootstrap.cc:654] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:56.892338 6749 ts_tablet_manager.cc:1397] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260808 14:42:56.892338 6750 ts_tablet_manager.cc:1397] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 14:42:56.893432 6752 log.cc:824] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:56.895607 6752 tablet_bootstrap.cc:492] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735: No bootstrap required, opened a new log
I20260808 14:42:56.895836 6752 ts_tablet_manager.cc:1397] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:42:56.896785 6750 raft_consensus.cc:348] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e892fdb075dc4a309b0583f93a41f0b6" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 40419 } } peers { permanent_uuid: "3ae4b13576ec4452b41f98d876b6e735" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35037 } } peers { permanent_uuid: "de8b6f76f89b4f2396f0e11e74361ff1" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34771 } }
I20260808 14:42:56.896889 6749 raft_consensus.cc:348] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e892fdb075dc4a309b0583f93a41f0b6" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 40419 } } peers { permanent_uuid: "3ae4b13576ec4452b41f98d876b6e735" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35037 } } peers { permanent_uuid: "de8b6f76f89b4f2396f0e11e74361ff1" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34771 } }
I20260808 14:42:56.897137 6749 raft_consensus.cc:374] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:56.897135 6750 raft_consensus.cc:374] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:56.897198 6749 raft_consensus.cc:729] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de8b6f76f89b4f2396f0e11e74361ff1, State: Initialized, Role: FOLLOWER
I20260808 14:42:56.897198 6750 raft_consensus.cc:729] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e892fdb075dc4a309b0583f93a41f0b6, State: Initialized, Role: FOLLOWER
I20260808 14:42:56.897770 6750 consensus_queue.cc:260] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e892fdb075dc4a309b0583f93a41f0b6" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 40419 } } peers { permanent_uuid: "3ae4b13576ec4452b41f98d876b6e735" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35037 } } peers { permanent_uuid: "de8b6f76f89b4f2396f0e11e74361ff1" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34771 } }
I20260808 14:42:56.897770 6749 consensus_queue.cc:260] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e892fdb075dc4a309b0583f93a41f0b6" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 40419 } } peers { permanent_uuid: "3ae4b13576ec4452b41f98d876b6e735" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35037 } } peers { permanent_uuid: "de8b6f76f89b4f2396f0e11e74361ff1" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34771 } }
I20260808 14:42:56.898844 6592 heartbeater.cc:503] Master 127.19.144.126:34351 was elected leader, sending a full tablet report...
I20260808 14:42:56.898877 6455 heartbeater.cc:503] Master 127.19.144.126:34351 was elected leader, sending a full tablet report...
I20260808 14:42:56.898983 6750 ts_tablet_manager.cc:1428] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 14:42:56.899505 6749 ts_tablet_manager.cc:1428] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260808 14:42:56.899868 6752 raft_consensus.cc:348] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e892fdb075dc4a309b0583f93a41f0b6" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 40419 } } peers { permanent_uuid: "3ae4b13576ec4452b41f98d876b6e735" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35037 } } peers { permanent_uuid: "de8b6f76f89b4f2396f0e11e74361ff1" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34771 } }
I20260808 14:42:56.900192 6752 raft_consensus.cc:374] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:56.900259 6752 raft_consensus.cc:729] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae4b13576ec4452b41f98d876b6e735, State: Initialized, Role: FOLLOWER
I20260808 14:42:56.900744 6752 consensus_queue.cc:260] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e892fdb075dc4a309b0583f93a41f0b6" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 40419 } } peers { permanent_uuid: "3ae4b13576ec4452b41f98d876b6e735" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35037 } } peers { permanent_uuid: "de8b6f76f89b4f2396f0e11e74361ff1" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34771 } }
I20260808 14:42:56.901538 6732 heartbeater.cc:503] Master 127.19.144.126:34351 was elected leader, sending a full tablet report...
I20260808 14:42:56.901839 6752 ts_tablet_manager.cc:1428] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 14:42:57.051602 6756 raft_consensus.cc:482] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:42:57.051859 6756 raft_consensus.cc:504] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e892fdb075dc4a309b0583f93a41f0b6" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 40419 } } peers { permanent_uuid: "3ae4b13576ec4452b41f98d876b6e735" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35037 } } peers { permanent_uuid: "de8b6f76f89b4f2396f0e11e74361ff1" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34771 } }
I20260808 14:42:57.053197 6756 leader_election.cc:288] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e892fdb075dc4a309b0583f93a41f0b6 (127.19.144.66:40419), 3ae4b13576ec4452b41f98d876b6e735 (127.19.144.67:35037)
I20260808 14:42:57.057962 6745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:57.053538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:54275 (local address 127.19.144.66:40419)
0808 14:42:57.053711 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:57.053717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:57.054055 (+ 338us) server_negotiation.cc:407] Connection header received
0808 14:42:57.054209 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:57.054213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:57.054273 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:57.054360 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:57.054869 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.055682 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.056871 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.057281 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.057408 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:57.057483 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:57.057731 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 14:42:57.057814 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 14:42:57.057974 6747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:57.053654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:38097 (local address 127.19.144.67:35037)
0808 14:42:57.053790 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:57.053795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:57.054055 (+ 260us) server_negotiation.cc:407] Connection header received
0808 14:42:57.054209 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:57.054213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:57.054270 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:57.054362 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:57.055005 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.055720 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.056845 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.057253 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.057398 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:57.057459 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:57.057764 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 14:42:57.057844 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 14:42:57.058038 6760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:57.053460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:40419 (local address 127.19.144.65:54275)
0808 14:42:57.053912 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:57.053928 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:57.054091 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:57.054389 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:57.054392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:57.054417 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:57.054739 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:57.054748 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.055845 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.055849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:57.056715 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:57.056726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.057398 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.057402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:57.057547 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:57.057569 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 14:42:57.057705 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 14:42:57.057800 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":130,"spinlock_wait_cycles":32512,"thread_start_us":153,"threads_started":1}
I20260808 14:42:57.058036 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:57.053542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:35037 (local address 127.19.144.65:38097)
0808 14:42:57.053963 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:57.053977 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:57.054091 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:57.054389 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:57.054392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:57.054418 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:57.054739 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:57.054748 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.055835 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.055841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:57.056710 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:57.056720 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.057400 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.057404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:57.057632 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:57.057668 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 14:42:57.057744 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 14:42:57.057828 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":81,"spinlock_wait_cycles":12672,"thread_start_us":141,"threads_started":1}
I20260808 14:42:57.058802 6686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97a95f6aa3be4df1984478d87f7152d6" candidate_uuid: "de8b6f76f89b4f2396f0e11e74361ff1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae4b13576ec4452b41f98d876b6e735" is_pre_election: true
I20260808 14:42:57.058776 6545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97a95f6aa3be4df1984478d87f7152d6" candidate_uuid: "de8b6f76f89b4f2396f0e11e74361ff1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e892fdb075dc4a309b0583f93a41f0b6" is_pre_election: true
I20260808 14:42:57.059149 6545 raft_consensus.cc:2454] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de8b6f76f89b4f2396f0e11e74361ff1 in term 0.
I20260808 14:42:57.059149 6686 raft_consensus.cc:2454] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de8b6f76f89b4f2396f0e11e74361ff1 in term 0.
I20260808 14:42:57.059710 6339 leader_election.cc:302] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3ae4b13576ec4452b41f98d876b6e735, de8b6f76f89b4f2396f0e11e74361ff1, e892fdb075dc4a309b0583f93a41f0b6; no voters:
I20260808 14:42:57.060009 6756 raft_consensus.cc:2781] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:42:57.060108 6756 raft_consensus.cc:482] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:42:57.060161 6756 raft_consensus.cc:3037] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:57.061347 6756 raft_consensus.cc:504] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e892fdb075dc4a309b0583f93a41f0b6" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 40419 } } peers { permanent_uuid: "3ae4b13576ec4452b41f98d876b6e735" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35037 } } peers { permanent_uuid: "de8b6f76f89b4f2396f0e11e74361ff1" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34771 } }
I20260808 14:42:57.061786 6756 leader_election.cc:288] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [CANDIDATE]: Term 1 election: Requested vote from peers e892fdb075dc4a309b0583f93a41f0b6 (127.19.144.66:40419), 3ae4b13576ec4452b41f98d876b6e735 (127.19.144.67:35037)
I20260808 14:42:57.062180 6686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97a95f6aa3be4df1984478d87f7152d6" candidate_uuid: "de8b6f76f89b4f2396f0e11e74361ff1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae4b13576ec4452b41f98d876b6e735"
I20260808 14:42:57.062180 6545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97a95f6aa3be4df1984478d87f7152d6" candidate_uuid: "de8b6f76f89b4f2396f0e11e74361ff1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e892fdb075dc4a309b0583f93a41f0b6"
I20260808 14:42:57.062326 6686 raft_consensus.cc:3037] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:57.062326 6545 raft_consensus.cc:3037] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:57.063759 6545 raft_consensus.cc:2454] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de8b6f76f89b4f2396f0e11e74361ff1 in term 1.
I20260808 14:42:57.063759 6686 raft_consensus.cc:2454] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de8b6f76f89b4f2396f0e11e74361ff1 in term 1.
W20260808 14:42:57.064127 6733 tablet.cc:2369] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:42:57.064184 6335 leader_election.cc:302] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae4b13576ec4452b41f98d876b6e735, de8b6f76f89b4f2396f0e11e74361ff1; no voters:
I20260808 14:42:57.064397 6756 raft_consensus.cc:2781] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:42:57.064703 6756 raft_consensus.cc:686] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [term 1 LEADER]: Becoming Leader. State: Replica: de8b6f76f89b4f2396f0e11e74361ff1, State: Running, Role: LEADER
I20260808 14:42:57.065090 6756 consensus_queue.cc:237] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e892fdb075dc4a309b0583f93a41f0b6" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 40419 } } peers { permanent_uuid: "3ae4b13576ec4452b41f98d876b6e735" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35037 } } peers { permanent_uuid: "de8b6f76f89b4f2396f0e11e74361ff1" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34771 } }
I20260808 14:42:57.068610 6254 catalog_manager.cc:5495] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 reported cstate change: term changed from 0 to 1, leader changed from <none> to de8b6f76f89b4f2396f0e11e74361ff1 (127.19.144.65). New cstate: current_term: 1 leader_uuid: "de8b6f76f89b4f2396f0e11e74361ff1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e892fdb075dc4a309b0583f93a41f0b6" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae4b13576ec4452b41f98d876b6e735" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de8b6f76f89b4f2396f0e11e74361ff1" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 34771 } health_report { overall_health: HEALTHY } } }
W20260808 14:42:57.072263 6593 tablet.cc:2369] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:42:57.089764 6456 tablet.cc:2369] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:42:57.110613 6748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:57.107081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56856 (local address 127.19.144.65:34771)
0808 14:42:57.107209 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:57.107213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:57.107227 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:42:57.107305 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:57.107307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:57.107383 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:57.107481 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:42:57.107719 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.108332 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.109205 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.109412 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.109497 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:57.109818 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:42:57.110255 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:42:57.110368 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:57.110460 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 14:42:57.110495 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 14:42:57.116855 6545 raft_consensus.cc:1261] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6 [term 1 FOLLOWER]: Refusing update from remote peer de8b6f76f89b4f2396f0e11e74361ff1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:42:57.116856 6686 raft_consensus.cc:1261] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735 [term 1 FOLLOWER]: Refusing update from remote peer de8b6f76f89b4f2396f0e11e74361ff1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:42:57.117656 6756 consensus_queue.cc:1035] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae4b13576ec4452b41f98d876b6e735" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 35037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:42:57.117910 6761 consensus_queue.cc:1035] T 97a95f6aa3be4df1984478d87f7152d6 P de8b6f76f89b4f2396f0e11e74361ff1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e892fdb075dc4a309b0583f93a41f0b6" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:42:57.126485 6765 mvcc.cc:205] Tried to move back new op lower bound from 7316275925462687744 to 7316275925261750272. Current Snapshot: MvccSnapshot[applied={T|T < 7316275925462687744}]
I20260808 14:42:57.138041 6254 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:42:57.138233 6254 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:42:57.140916 6254 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bbb99efbaf04b789a6fb66f5ef93c9f: Sending DeleteTablet for 3 replicas of tablet 97a95f6aa3be4df1984478d87f7152d6
I20260808 14:42:57.141842 6525 tablet_service.cc:1514] Processing DeleteTablet for tablet 97a95f6aa3be4df1984478d87f7152d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:42:57 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:38528
I20260808 14:42:57.141834 6664 tablet_service.cc:1514] Processing DeleteTablet for tablet 97a95f6aa3be4df1984478d87f7152d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:42:57 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:46772
I20260808 14:42:57.142455 6776 tablet_replica.cc:330] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6: stopping tablet replica
I20260808 14:42:57.142748 6386 tablet_service.cc:1514] Processing DeleteTablet for tablet 97a95f6aa3be4df1984478d87f7152d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:42:57 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:56846
I20260808 14:42:57.142831 6776 raft_consensus.cc:2229] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:42:57.143210 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 6319
I20260808 14:42:57.143220 6776 raft_consensus.cc:2258] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:42:57.143633 6775 tablet_replica.cc:330] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735: stopping tablet replica
I20260808 14:42:57.143975 6775 raft_consensus.cc:2229] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:42:57.144287 6775 raft_consensus.cc:2258] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:42:57.146654 6775 ts_tablet_manager.cc:1905] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:42:57.146921 6776 ts_tablet_manager.cc:1905] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:42:57.149163 6776 ts_tablet_manager.cc:1918] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:42:57.149296 6776 log.cc:1196] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/97a95f6aa3be4df1984478d87f7152d6
I20260808 14:42:57.149592 6776 ts_tablet_manager.cc:1939] T 97a95f6aa3be4df1984478d87f7152d6 P e892fdb075dc4a309b0583f93a41f0b6: Deleting consensus metadata
I20260808 14:42:57.150358 6775 ts_tablet_manager.cc:1918] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:42:57.150480 6775 log.cc:1196] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/97a95f6aa3be4df1984478d87f7152d6
I20260808 14:42:57.150848 6775 ts_tablet_manager.cc:1939] T 97a95f6aa3be4df1984478d87f7152d6 P 3ae4b13576ec4452b41f98d876b6e735: Deleting consensus metadata
I20260808 14:42:57.150987 6242 catalog_manager.cc:4841] TS e892fdb075dc4a309b0583f93a41f0b6 (127.19.144.66:40419): tablet 97a95f6aa3be4df1984478d87f7152d6 (table test-table [id=d9b571dc86ab4999acb6abab047248a9]) successfully deleted
I20260808 14:42:57.152182 6239 catalog_manager.cc:4841] TS 3ae4b13576ec4452b41f98d876b6e735 (127.19.144.67:35037): tablet 97a95f6aa3be4df1984478d87f7152d6 (table test-table [id=d9b571dc86ab4999acb6abab047248a9]) successfully deleted
W20260808 14:42:57.153671 6240 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.144.65:34771 (error 108)
I20260808 14:42:57.153939 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 6460
I20260808 14:42:57.157009 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:57.156615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:34771 (local address 127.0.0.1:56870)
0808 14:42:57.156818 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:57.156893 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:34771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260808 14:42:57.157208 6240 catalog_manager.cc:4584] TS de8b6f76f89b4f2396f0e11e74361ff1 (127.19.144.65:34771): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 97a95f6aa3be4df1984478d87f7152d6: Network error: Client connection negotiation failed: client connection to 127.19.144.65:34771: connect: Connection refused (error 111)
I20260808 14:42:57.163654 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 6597
I20260808 14:42:57.171161 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 6223
2026-08-08T14:42:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2552 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T14:42:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:42:57Z Disabled control of system clock
I20260808 14:42:57.205276 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:44611
--webserver_interface=127.19.144.126
--webserver_port=0
--builtin_ntp_servers=127.19.144.84:37983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:44611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 14:42:57.322535 6782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:57.322842 6782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:57.322880 6782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:57.326972 6782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:42:57.327033 6782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:57.327049 6782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:57.327066 6782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:42:57.327082 6782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:42:57.332098 6782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:37983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:44611
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:44611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:57.333235 6782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:57.334702 6782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:57.343808 6788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:57.343842 6790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:57.343842 6787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:57.344246 6782 server_base.cc:1034] running on GCE node
I20260808 14:42:57.344856 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:57.346148 6782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:57.347337 6782 hybrid_clock.cc:648] HybridClock initialized: now 1786200177347300 us; error 51 us; skew 500 ppm
I20260808 14:42:57.349308 6782 webserver.cc:460] Webserver started at http://127.19.144.126:32909/ using document root <none> and password file <none>
I20260808 14:42:57.349988 6782 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:57.350083 6782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:57.350324 6782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:57.352527 6782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "1a97e064a4cf4a21ae4ff9b9ddd6ce29"
format_stamp: "Formatted at 2026-08-08 14:42:57 on dist-test-slave-p3t5"
server_key: "ee19137be9396871f0cc4c0f9dd76154"
server_key_iv: "fac2f23cdc07c459460d7ccdbe75077e"
server_key_version: "encryptionkey@0"
I20260808 14:42:57.353124 6782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "1a97e064a4cf4a21ae4ff9b9ddd6ce29"
format_stamp: "Formatted at 2026-08-08 14:42:57 on dist-test-slave-p3t5"
server_key: "ee19137be9396871f0cc4c0f9dd76154"
server_key_iv: "fac2f23cdc07c459460d7ccdbe75077e"
server_key_version: "encryptionkey@0"
I20260808 14:42:57.357110 6782 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:42:57.359843 6796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:57.361217 6782 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:42:57.361370 6782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "1a97e064a4cf4a21ae4ff9b9ddd6ce29"
format_stamp: "Formatted at 2026-08-08 14:42:57 on dist-test-slave-p3t5"
server_key: "ee19137be9396871f0cc4c0f9dd76154"
server_key_iv: "fac2f23cdc07c459460d7ccdbe75077e"
server_key_version: "encryptionkey@0"
I20260808 14:42:57.361505 6782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:57.380340 6782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:57.381141 6782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:57.381368 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:57.388613 6782 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:44611
I20260808 14:42:57.388625 6849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:44611 every 8 connection(s)
I20260808 14:42:57.389664 6782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:42:57.392329 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 6782
I20260808 14:42:57.392488 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:42:57.392781 20033 external_mini_cluster.cc:1428] Setting key c4333951c313425bdae66625b7fd4b7e
I20260808 14:42:57.393163 6850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:57.401001 6850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29: Bootstrap starting.
I20260808 14:42:57.403201 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:57.394272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40806 (local address 127.19.144.126:44611)
0808 14:42:57.394942 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:57.394959 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:57.394996 (+ 37us) server_negotiation.cc:407] Connection header received
0808 14:42:57.396020 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:57.396041 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:57.396499 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:57.396807 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:57.397247 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.398347 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.399666 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.400163 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.400288 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:57.401286 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:57.401292 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:57.401306 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:42:57.401802 (+ 496us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:57.401863 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:42:57.401884 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:57.401965 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:57.402127 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 14:42:57.402440 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":154,"threads_started":1}
I20260808 14:42:57.403650 6850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:57.404443 6850 log.cc:824] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:57.406757 6850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29: No bootstrap required, opened a new log
I20260808 14:42:57.412267 6850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a97e064a4cf4a21ae4ff9b9ddd6ce29" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 44611 } }
I20260808 14:42:57.412602 6850 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:57.412675 6850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a97e064a4cf4a21ae4ff9b9ddd6ce29, State: Initialized, Role: FOLLOWER
I20260808 14:42:57.413203 6850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a97e064a4cf4a21ae4ff9b9ddd6ce29" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 44611 } }
I20260808 14:42:57.413369 6850 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:42:57.413450 6850 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:42:57.413564 6850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:57.414702 6850 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a97e064a4cf4a21ae4ff9b9ddd6ce29" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 44611 } }
I20260808 14:42:57.415113 6850 leader_election.cc:302] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a97e064a4cf4a21ae4ff9b9ddd6ce29; no voters:
I20260808 14:42:57.415491 6850 leader_election.cc:288] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:42:57.415632 6855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:42:57.415910 6855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [term 1 LEADER]: Becoming Leader. State: Replica: 1a97e064a4cf4a21ae4ff9b9ddd6ce29, State: Running, Role: LEADER
I20260808 14:42:57.416249 6855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a97e064a4cf4a21ae4ff9b9ddd6ce29" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 44611 } }
I20260808 14:42:57.416953 6850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:42:57.417965 6857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a97e064a4cf4a21ae4ff9b9ddd6ce29. Latest consensus state: current_term: 1 leader_uuid: "1a97e064a4cf4a21ae4ff9b9ddd6ce29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a97e064a4cf4a21ae4ff9b9ddd6ce29" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 44611 } } }
I20260808 14:42:57.418128 6857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [sys.catalog]: This master's current role is: LEADER
I20260808 14:42:57.418644 6856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a97e064a4cf4a21ae4ff9b9ddd6ce29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a97e064a4cf4a21ae4ff9b9ddd6ce29" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 44611 } } }
I20260808 14:42:57.418787 6856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29 [sys.catalog]: This master's current role is: LEADER
W20260808 14:42:57.422567 6871 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:42:57.422664 6871 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:42:57.422729 6863 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:42:57.423403 6863 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:42:57.429746 6863 catalog_manager.cc:1347] Generated new cluster ID: a2aa009c99ed4b679e6ccc8f2ab03552
I20260808 14:42:57.429827 6863 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:42:57.441270 6863 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:42:57.442323 6863 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:42:57.450600 6863 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29: Generated new TSK 0
I20260808 14:42:57.451354 6863 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:42:57.461076 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:44611
--builtin_ntp_servers=127.19.144.84:37983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 14:42:57.579800 6875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:57.580066 6875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:57.580111 6875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:57.584162 6875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:57.584223 6875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:57.584278 6875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:42:57.589182 6875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:37983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:44611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:57.590400 6875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:57.591861 6875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:57.602015 6883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:57.602034 6881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:57.602174 6875 server_base.cc:1034] running on GCE node
W20260808 14:42:57.602015 6880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:57.602794 6875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:57.603510 6875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:57.604707 6875 hybrid_clock.cc:648] HybridClock initialized: now 1786200177604685 us; error 35 us; skew 500 ppm
I20260808 14:42:57.606833 6875 webserver.cc:460] Webserver started at http://127.19.144.65:37515/ using document root <none> and password file <none>
I20260808 14:42:57.607473 6875 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:57.607569 6875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:57.607810 6875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:57.610216 6875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "0c39635bca214f319d45db090ea34993"
format_stamp: "Formatted at 2026-08-08 14:42:57 on dist-test-slave-p3t5"
server_key: "574849f2e8d92ea4dfd790a60a27cd14"
server_key_iv: "102e2eb7fd01cd3c9b69a3c678e49104"
server_key_version: "encryptionkey@0"
I20260808 14:42:57.610837 6875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "0c39635bca214f319d45db090ea34993"
format_stamp: "Formatted at 2026-08-08 14:42:57 on dist-test-slave-p3t5"
server_key: "574849f2e8d92ea4dfd790a60a27cd14"
server_key_iv: "102e2eb7fd01cd3c9b69a3c678e49104"
server_key_version: "encryptionkey@0"
I20260808 14:42:57.614912 6875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 14:42:57.617645 6889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:57.619086 6875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:42:57.619227 6875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "0c39635bca214f319d45db090ea34993"
format_stamp: "Formatted at 2026-08-08 14:42:57 on dist-test-slave-p3t5"
server_key: "574849f2e8d92ea4dfd790a60a27cd14"
server_key_iv: "102e2eb7fd01cd3c9b69a3c678e49104"
server_key_version: "encryptionkey@0"
I20260808 14:42:57.619349 6875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:57.630515 6875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:57.631268 6875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:57.631474 6875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:57.632077 6875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:57.633157 6875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:42:57.633222 6875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:57.633276 6875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:42:57.633307 6875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:57.642324 6875 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:42637
I20260808 14:42:57.642351 7005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:42637 every 8 connection(s)
I20260808 14:42:57.643292 6875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:42:57.649240 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 6875
I20260808 14:42:57.649396 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:42:57.649778 20033 external_mini_cluster.cc:1428] Setting key 7d6263d8c2f3048ef5fdba8c200de73e
I20260808 14:42:57.651963 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:44611
--builtin_ntp_servers=127.19.144.84:37983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 14:42:57.653467 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:57.645168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:49063 (local address 127.19.144.126:44611)
0808 14:42:57.645322 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:57.645327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:57.646564 (+ 1237us) server_negotiation.cc:407] Connection header received
0808 14:42:57.647992 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:57.647995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:57.648044 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:57.648124 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:57.649135 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.649838 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.651320 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.651541 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.651625 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:57.652991 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:57.652994 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:57.652996 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:42:57.653009 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:57.653029 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:42:57.653037 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:57.653093 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:57.653280 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 14:42:57.653348 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:42:57.654620 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:57.645719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:44611 (local address 127.19.144.65:49063)
0808 14:42:57.646400 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:57.646446 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:57.647776 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:57.648305 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:57.648312 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:57.648401 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:57.648960 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:57.648984 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.649985 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.649988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:57.651181 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:57.651191 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.651713 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.651720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:57.651960 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:57.652646 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:42:57.652651 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:57.652713 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:42:57.652809 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:57.653118 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:57.653130 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:57.653140 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 14:42:57.653278 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 14:42:57.653642 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":125,"threads_started":1}
I20260808 14:42:57.656019 7006 heartbeater.cc:347] Connected to a master server at 127.19.144.126:44611
I20260808 14:42:57.656284 7006 heartbeater.cc:464] Registering TS with master...
I20260808 14:42:57.656852 7006 heartbeater.cc:511] Master 127.19.144.126:44611 requested a full tablet report, sending...
I20260808 14:42:57.658589 6814 ts_manager.cc:194] Registered new tserver with Master: 0c39635bca214f319d45db090ea34993 (127.19.144.65:42637)
I20260808 14:42:57.659787 6814 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.144.65:49063
W20260808 14:42:57.771301 7009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:57.771654 7009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:57.771731 7009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:57.776000 7009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:57.776093 7009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:57.776194 7009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:42:57.781186 7009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:37983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:44611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:57.782506 7009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:57.784026 7009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:57.794168 7014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:57.794144 7015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:57.794140 7017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:57.794390 7009 server_base.cc:1034] running on GCE node
I20260808 14:42:57.794929 7009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:57.795655 7009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:57.796846 7009 hybrid_clock.cc:648] HybridClock initialized: now 1786200177796820 us; error 36 us; skew 500 ppm
I20260808 14:42:57.798873 7009 webserver.cc:460] Webserver started at http://127.19.144.66:39689/ using document root <none> and password file <none>
I20260808 14:42:57.799500 7009 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:57.799599 7009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:57.799842 7009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:57.802121 7009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "d7c5cb6f8d434f7c9e54c7a364c9852b"
format_stamp: "Formatted at 2026-08-08 14:42:57 on dist-test-slave-p3t5"
server_key: "e0de69a119f991ec6ae017021787b960"
server_key_iv: "7afd0f6cf6af50c5e3c97b5cb0039145"
server_key_version: "encryptionkey@0"
I20260808 14:42:57.802731 7009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "d7c5cb6f8d434f7c9e54c7a364c9852b"
format_stamp: "Formatted at 2026-08-08 14:42:57 on dist-test-slave-p3t5"
server_key: "e0de69a119f991ec6ae017021787b960"
server_key_iv: "7afd0f6cf6af50c5e3c97b5cb0039145"
server_key_version: "encryptionkey@0"
I20260808 14:42:57.806730 7009 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:42:57.809417 7023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:57.810753 7009 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:42:57.810897 7009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "d7c5cb6f8d434f7c9e54c7a364c9852b"
format_stamp: "Formatted at 2026-08-08 14:42:57 on dist-test-slave-p3t5"
server_key: "e0de69a119f991ec6ae017021787b960"
server_key_iv: "7afd0f6cf6af50c5e3c97b5cb0039145"
server_key_version: "encryptionkey@0"
I20260808 14:42:57.811038 7009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:57.842219 7009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:57.842996 7009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:57.843209 7009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:57.843847 7009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:57.844869 7009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:42:57.844949 7009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:57.845026 7009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:42:57.845077 7009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:57.854280 7009 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:44723
I20260808 14:42:57.854311 7141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:44723 every 8 connection(s)
I20260808 14:42:57.855262 7009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:42:57.860745 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 7009
I20260808 14:42:57.860893 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:42:57.861236 20033 external_mini_cluster.cc:1428] Setting key caf4438b33d3bbc640ca3d283dad934a
I20260808 14:42:57.863198 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:44611
--builtin_ntp_servers=127.19.144.84:37983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 14:42:57.864941 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:57.857118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:39769 (local address 127.19.144.126:44611)
0808 14:42:57.857273 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:57.857279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:57.858473 (+ 1194us) server_negotiation.cc:407] Connection header received
0808 14:42:57.859831 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:57.859834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:57.859885 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:57.859966 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:57.861092 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.861721 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.862894 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.863119 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.863229 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:57.864414 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:57.864417 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:57.864420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:42:57.864443 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:57.864463 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:42:57.864471 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:57.864546 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:57.864734 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 14:42:57.864799 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:42:57.865998 7144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:57.857588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:44611 (local address 127.19.144.66:39769)
0808 14:42:57.858303 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:57.858345 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:57.859629 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:57.860138 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:57.860145 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:57.860232 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:57.860936 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:57.860960 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.861846 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.861849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:57.862765 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:57.862775 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:57.863274 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:57.863278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:57.863483 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:57.864140 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:42:57.864141 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:57.864184 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:42:57.864255 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:57.864562 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:57.864577 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:57.864583 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 14:42:57.864734 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 14:42:57.865057 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":119,"threads_started":1}
I20260808 14:42:57.867169 7142 heartbeater.cc:347] Connected to a master server at 127.19.144.126:44611
I20260808 14:42:57.867431 7142 heartbeater.cc:464] Registering TS with master...
I20260808 14:42:57.868105 7142 heartbeater.cc:511] Master 127.19.144.126:44611 requested a full tablet report, sending...
I20260808 14:42:57.869350 6814 ts_manager.cc:194] Registered new tserver with Master: d7c5cb6f8d434f7c9e54c7a364c9852b (127.19.144.66:44723)
I20260808 14:42:57.870288 6814 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.144.66:39769
W20260808 14:42:57.982699 7145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:57.983000 7145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:57.983042 7145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:57.987090 7145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:57.987156 7145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:57.987211 7145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:42:57.992102 7145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:37983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:44611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:57.993178 7145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:57.995030 7145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:58.005169 7154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:58.005172 7151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:58.005173 7150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:58.005517 7145 server_base.cc:1034] running on GCE node
I20260808 14:42:58.006076 7145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:58.006789 7145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:58.008005 7145 hybrid_clock.cc:648] HybridClock initialized: now 1786200178007980 us; error 42 us; skew 500 ppm
I20260808 14:42:58.010111 7145 webserver.cc:460] Webserver started at http://127.19.144.67:45671/ using document root <none> and password file <none>
I20260808 14:42:58.010733 7145 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:58.010831 7145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:58.011073 7145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:58.013229 7145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca"
format_stamp: "Formatted at 2026-08-08 14:42:58 on dist-test-slave-p3t5"
server_key: "f5c4a678ef4cd948dad5019bf0551af7"
server_key_iv: "6391d36fda2471c6a9f247eb17f94040"
server_key_version: "encryptionkey@0"
I20260808 14:42:58.013880 7145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca"
format_stamp: "Formatted at 2026-08-08 14:42:58 on dist-test-slave-p3t5"
server_key: "f5c4a678ef4cd948dad5019bf0551af7"
server_key_iv: "6391d36fda2471c6a9f247eb17f94040"
server_key_version: "encryptionkey@0"
I20260808 14:42:58.017980 7145 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:42:58.020538 7160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:58.021848 7145 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:42:58.021989 7145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca"
format_stamp: "Formatted at 2026-08-08 14:42:58 on dist-test-slave-p3t5"
server_key: "f5c4a678ef4cd948dad5019bf0551af7"
server_key_iv: "6391d36fda2471c6a9f247eb17f94040"
server_key_version: "encryptionkey@0"
I20260808 14:42:58.022102 7145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:42:58.052740 7145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:42:58.053555 7145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:42:58.053809 7145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:42:58.054466 7145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:42:58.055513 7145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:42:58.055584 7145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:58.055656 7145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:42:58.055701 7145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:42:58.064991 7145 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:39185
I20260808 14:42:58.065013 7279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:39185 every 8 connection(s)
I20260808 14:42:58.066159 7145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:42:58.072098 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 7145
I20260808 14:42:58.072228 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:42:58.072533 20033 external_mini_cluster.cc:1428] Setting key dfee8c52c566f362f0ff2bb1da7f30dd
I20260808 14:42:58.076408 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:58.068146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:52547 (local address 127.19.144.126:44611)
0808 14:42:58.068284 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:58.068288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:58.069442 (+ 1154us) server_negotiation.cc:407] Connection header received
0808 14:42:58.070879 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:58.070882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:58.070937 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:58.071014 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:58.072143 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.072903 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.074453 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.074813 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.074899 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:58.076023 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:58.076025 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:58.076027 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:42:58.076038 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:58.076055 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:42:58.076062 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:58.076115 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:58.076233 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 14:42:58.076282 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 14:42:58.077286 7282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:58.068573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:44611 (local address 127.19.144.67:52547)
0808 14:42:58.069285 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:58.069330 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:58.070672 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:58.071324 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:58.071331 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:58.071424 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:42:58.071977 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:58.071993 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.073039 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.073043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:58.074324 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:58.074335 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.074943 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.074946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:58.075105 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:58.075763 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:42:58.075765 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:42:58.075810 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:42:58.075888 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:42:58.076114 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:42:58.076118 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:42:58.076122 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 14:42:58.076215 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 14:42:58.076447 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":123,"threads_started":1}
I20260808 14:42:58.078441 7280 heartbeater.cc:347] Connected to a master server at 127.19.144.126:44611
I20260808 14:42:58.078689 7280 heartbeater.cc:464] Registering TS with master...
I20260808 14:42:58.079226 7280 heartbeater.cc:511] Master 127.19.144.126:44611 requested a full tablet report, sending...
I20260808 14:42:58.080230 6814 ts_manager.cc:194] Registered new tserver with Master: 1fcf906f93dc4f65bf69d2f8ebc65eca (127.19.144.67:39185)
I20260808 14:42:58.080792 6814 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.144.67:52547
I20260808 14:42:58.086359 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:42:58.092025 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:58.088364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40820 (local address 127.19.144.126:44611)
0808 14:42:58.088489 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:58.088493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:58.088670 (+ 177us) server_negotiation.cc:407] Connection header received
0808 14:42:58.088769 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:58.088771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:58.088810 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:58.088878 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:42:58.089121 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.089604 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.090410 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.090614 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.090690 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:58.091622 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:42:58.091624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:42:58.091626 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:42:58.091637 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:42:58.091654 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:42:58.091660 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:42:58.091748 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:58.091874 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 14:42:58.091905 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:42:58.094946 6814 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:42:58.097926 6814 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:42:58.112826 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:58.106670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:42637 (local address 127.0.0.1:33130)
0808 14:42:58.107482 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:58.107537 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:58.107675 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:58.108354 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:58.108360 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:58.108389 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:58.108550 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:58.108558 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.109886 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.109889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:58.111339 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:58.111350 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.112241 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.112245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:58.112367 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:58.112409 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 14:42:58.112528 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 14:42:58.112663 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":111,"threads_started":1}
I20260808 14:42:58.112846 7295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:58.107441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33130 (local address 127.19.144.65:42637)
0808 14:42:58.107808 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:58.107821 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:58.107863 (+ 42us) server_negotiation.cc:407] Connection header received
0808 14:42:58.107953 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:58.107961 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:58.108199 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:58.108370 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:58.108676 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.109772 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.111471 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.112110 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.112240 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:58.112452 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:58.112561 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 14:42:58.112663 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":138,"threads_started":1}
I20260808 14:42:58.113291 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:58.106860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:44723 (local address 127.0.0.1:47326)
0808 14:42:58.108133 (+ 1273us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:58.108147 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:58.108250 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:58.109160 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:58.109163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:58.109179 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:58.109274 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:58.109282 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.110521 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.110525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:58.111539 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:58.111549 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.112750 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.112754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:58.112929 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:58.113051 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 14:42:58.113112 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 14:42:58.113159 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1218,"mutex_wait_us":53,"thread_start_us":106,"threads_started":1}
I20260808 14:42:58.113291 7292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:58.106670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:39185 (local address 127.0.0.1:49942)
0808 14:42:58.108443 (+ 1773us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:58.108456 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:58.108548 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:58.109004 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:58.109007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:58.109027 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:58.109127 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:58.109133 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.110331 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.110334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:58.111481 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:58.111489 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.112750 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.112754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:58.112929 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:58.112966 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 14:42:58.113047 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 14:42:58.113134 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1712,"mutex_wait_us":1221,"thread_start_us":102,"threads_started":1}
I20260808 14:42:58.113370 7296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:58.107493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49942 (local address 127.19.144.67:39185)
0808 14:42:58.108357 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:58.108369 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:58.108557 (+ 188us) server_negotiation.cc:407] Connection header received
0808 14:42:58.108643 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:58.108650 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:58.108852 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:58.109013 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:58.109231 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.110198 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.111592 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.112388 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.112499 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:58.112681 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:58.113091 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 14:42:58.113193 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":181,"threads_started":1}
I20260808 14:42:58.114701 7294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:58.107016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47326 (local address 127.19.144.66:44723)
0808 14:42:58.107993 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:58.108014 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:58.108237 (+ 223us) server_negotiation.cc:407] Connection header received
0808 14:42:58.108731 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:58.108736 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:58.108880 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:58.109021 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:58.109402 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.110390 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.111647 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.112388 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.112499 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:58.112681 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:58.114431 (+ 1750us) server_negotiation.cc:299] Negotiation successful
0808 14:42:58.114536 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":843,"thread_start_us":124,"threads_started":1}
I20260808 14:42:58.116425 6939 tablet_service.cc:1467] Processing CreateTablet for tablet ae0ce430212e41a5af7ffd40f1d360ab (DEFAULT_TABLE table=test-table [id=b1c6a77addf24d87892770a5c54b978d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:42:58.116417 7211 tablet_service.cc:1467] Processing CreateTablet for tablet ae0ce430212e41a5af7ffd40f1d360ab (DEFAULT_TABLE table=test-table [id=b1c6a77addf24d87892770a5c54b978d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:42:58.117578 6939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae0ce430212e41a5af7ffd40f1d360ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:58.117563 7211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae0ce430212e41a5af7ffd40f1d360ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:58.117446 7071 tablet_service.cc:1467] Processing CreateTablet for tablet ae0ce430212e41a5af7ffd40f1d360ab (DEFAULT_TABLE table=test-table [id=b1c6a77addf24d87892770a5c54b978d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:42:58.118541 7071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae0ce430212e41a5af7ffd40f1d360ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:42:58.124730 7298 tablet_bootstrap.cc:492] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca: Bootstrap starting.
I20260808 14:42:58.125980 7299 tablet_bootstrap.cc:492] T ae0ce430212e41a5af7ffd40f1d360ab P 0c39635bca214f319d45db090ea34993: Bootstrap starting.
I20260808 14:42:58.126541 7300 tablet_bootstrap.cc:492] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b: Bootstrap starting.
I20260808 14:42:58.127228 7298 tablet_bootstrap.cc:654] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:58.128095 7299 tablet_bootstrap.cc:654] T ae0ce430212e41a5af7ffd40f1d360ab P 0c39635bca214f319d45db090ea34993: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:58.128170 7298 log.cc:824] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:58.128793 7300 tablet_bootstrap.cc:654] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b: Neither blocks nor log segments found. Creating new log.
I20260808 14:42:58.128963 7299 log.cc:824] T ae0ce430212e41a5af7ffd40f1d360ab P 0c39635bca214f319d45db090ea34993: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:58.129652 7300 log.cc:824] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b: Log is configured to *not* fsync() on all Append() calls
I20260808 14:42:58.130872 7298 tablet_bootstrap.cc:492] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca: No bootstrap required, opened a new log
I20260808 14:42:58.131088 7298 ts_tablet_manager.cc:1397] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 14:42:58.131672 7300 tablet_bootstrap.cc:492] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b: No bootstrap required, opened a new log
I20260808 14:42:58.131870 7300 ts_tablet_manager.cc:1397] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 14:42:58.135740 7298 raft_consensus.cc:348] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39185 } } peers { permanent_uuid: "0c39635bca214f319d45db090ea34993" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 42637 } } peers { permanent_uuid: "d7c5cb6f8d434f7c9e54c7a364c9852b" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44723 } }
I20260808 14:42:58.136067 7298 raft_consensus.cc:374] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:58.136142 7298 raft_consensus.cc:729] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fcf906f93dc4f65bf69d2f8ebc65eca, State: Initialized, Role: FOLLOWER
I20260808 14:42:58.136157 7300 raft_consensus.cc:348] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39185 } } peers { permanent_uuid: "0c39635bca214f319d45db090ea34993" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 42637 } } peers { permanent_uuid: "d7c5cb6f8d434f7c9e54c7a364c9852b" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44723 } }
I20260808 14:42:58.136426 7300 raft_consensus.cc:374] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:58.136598 7300 raft_consensus.cc:729] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7c5cb6f8d434f7c9e54c7a364c9852b, State: Initialized, Role: FOLLOWER
I20260808 14:42:58.136658 7298 consensus_queue.cc:260] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39185 } } peers { permanent_uuid: "0c39635bca214f319d45db090ea34993" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 42637 } } peers { permanent_uuid: "d7c5cb6f8d434f7c9e54c7a364c9852b" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44723 } }
I20260808 14:42:58.137141 7300 consensus_queue.cc:260] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39185 } } peers { permanent_uuid: "0c39635bca214f319d45db090ea34993" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 42637 } } peers { permanent_uuid: "d7c5cb6f8d434f7c9e54c7a364c9852b" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44723 } }
I20260808 14:42:58.137892 7298 ts_tablet_manager.cc:1428] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260808 14:42:58.138127 7300 ts_tablet_manager.cc:1428] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 14:42:58.138329 7280 heartbeater.cc:503] Master 127.19.144.126:44611 was elected leader, sending a full tablet report...
I20260808 14:42:58.138468 7142 heartbeater.cc:503] Master 127.19.144.126:44611 was elected leader, sending a full tablet report...
I20260808 14:42:58.140575 7299 tablet_bootstrap.cc:492] T ae0ce430212e41a5af7ffd40f1d360ab P 0c39635bca214f319d45db090ea34993: No bootstrap required, opened a new log
I20260808 14:42:58.140815 7299 ts_tablet_manager.cc:1397] T ae0ce430212e41a5af7ffd40f1d360ab P 0c39635bca214f319d45db090ea34993: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
I20260808 14:42:58.144935 7299 raft_consensus.cc:348] T ae0ce430212e41a5af7ffd40f1d360ab P 0c39635bca214f319d45db090ea34993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39185 } } peers { permanent_uuid: "0c39635bca214f319d45db090ea34993" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 42637 } } peers { permanent_uuid: "d7c5cb6f8d434f7c9e54c7a364c9852b" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44723 } }
I20260808 14:42:58.145169 7299 raft_consensus.cc:374] T ae0ce430212e41a5af7ffd40f1d360ab P 0c39635bca214f319d45db090ea34993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:42:58.145203 7299 raft_consensus.cc:729] T ae0ce430212e41a5af7ffd40f1d360ab P 0c39635bca214f319d45db090ea34993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c39635bca214f319d45db090ea34993, State: Initialized, Role: FOLLOWER
I20260808 14:42:58.145660 7299 consensus_queue.cc:260] T ae0ce430212e41a5af7ffd40f1d360ab P 0c39635bca214f319d45db090ea34993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39185 } } peers { permanent_uuid: "0c39635bca214f319d45db090ea34993" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 42637 } } peers { permanent_uuid: "d7c5cb6f8d434f7c9e54c7a364c9852b" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44723 } }
I20260808 14:42:58.146571 7006 heartbeater.cc:503] Master 127.19.144.126:44611 was elected leader, sending a full tablet report...
I20260808 14:42:58.146919 7299 ts_tablet_manager.cc:1428] T ae0ce430212e41a5af7ffd40f1d360ab P 0c39635bca214f319d45db090ea34993: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 14:42:58.317274 7281 tablet.cc:2369] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:42:58.356462 7143 tablet.cc:2369] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 14:42:58.394672 7007 tablet.cc:2369] T ae0ce430212e41a5af7ffd40f1d360ab P 0c39635bca214f319d45db090ea34993: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:42:58.468742 7304 raft_consensus.cc:482] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:42:58.468999 7304 raft_consensus.cc:504] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39185 } } peers { permanent_uuid: "0c39635bca214f319d45db090ea34993" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 42637 } } peers { permanent_uuid: "d7c5cb6f8d434f7c9e54c7a364c9852b" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44723 } }
I20260808 14:42:58.470371 7304 leader_election.cc:288] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c39635bca214f319d45db090ea34993 (127.19.144.65:42637), d7c5cb6f8d434f7c9e54c7a364c9852b (127.19.144.66:44723)
I20260808 14:42:58.474406 7294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:58.471125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:39611 (local address 127.19.144.66:44723)
0808 14:42:58.471249 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:58.471253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:58.471267 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:42:58.471319 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:58.471321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:58.471370 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:58.471477 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:58.471802 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.472334 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.473384 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.473853 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.473980 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:58.474038 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:58.474223 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 14:42:58.474292 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:42:58.474407 7282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:58.470601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:42637 (local address 127.19.144.67:42571)
0808 14:42:58.470875 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:58.470895 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:58.471004 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:58.471327 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:58.471330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:58.471345 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:58.471473 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:58.471480 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.472305 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.472309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:58.473184 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:58.473195 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.473894 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.473900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:58.474040 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:58.474070 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 14:42:58.474140 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 14:42:58.474235 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":23}
I20260808 14:42:58.474403 7295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:58.470822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:42571 (local address 127.19.144.65:42637)
0808 14:42:58.470975 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:58.470980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:58.471000 (+ 20us) server_negotiation.cc:407] Connection header received
0808 14:42:58.471161 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:58.471166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:58.471228 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:58.471310 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:42:58.471609 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.472188 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.473334 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.473779 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.473874 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:58.473947 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:58.474182 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 14:42:58.474259 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 14:42:58.474419 7307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:58.470684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:44723 (local address 127.19.144.67:39611)
0808 14:42:58.471070 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:58.471084 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:42:58.471160 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:42:58.471478 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:42:58.471481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:42:58.471495 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:42:58.471633 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:58.471639 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.472459 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.472461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:58.473228 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:42:58.473238 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.473982 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.473985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:42:58.474081 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:58.474098 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 14:42:58.474192 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 14:42:58.474258 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":1,"thread_start_us":113,"threads_started":1}
I20260808 14:42:58.475135 6960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0ce430212e41a5af7ffd40f1d360ab" candidate_uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c39635bca214f319d45db090ea34993" is_pre_election: true
I20260808 14:42:58.475366 7092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0ce430212e41a5af7ffd40f1d360ab" candidate_uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c5cb6f8d434f7c9e54c7a364c9852b" is_pre_election: true
I20260808 14:42:58.475485 6960 raft_consensus.cc:2454] T ae0ce430212e41a5af7ffd40f1d360ab P 0c39635bca214f319d45db090ea34993 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fcf906f93dc4f65bf69d2f8ebc65eca in term 0.
I20260808 14:42:58.475740 7092 raft_consensus.cc:2454] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fcf906f93dc4f65bf69d2f8ebc65eca in term 0.
I20260808 14:42:58.476109 7165 leader_election.cc:302] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c39635bca214f319d45db090ea34993, 1fcf906f93dc4f65bf69d2f8ebc65eca; no voters:
I20260808 14:42:58.476405 7304 raft_consensus.cc:2781] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:42:58.476497 7304 raft_consensus.cc:482] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:42:58.476543 7304 raft_consensus.cc:3037] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:58.477581 7304 raft_consensus.cc:504] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39185 } } peers { permanent_uuid: "0c39635bca214f319d45db090ea34993" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 42637 } } peers { permanent_uuid: "d7c5cb6f8d434f7c9e54c7a364c9852b" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44723 } }
I20260808 14:42:58.477981 7304 leader_election.cc:288] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [CANDIDATE]: Term 1 election: Requested vote from peers 0c39635bca214f319d45db090ea34993 (127.19.144.65:42637), d7c5cb6f8d434f7c9e54c7a364c9852b (127.19.144.66:44723)
I20260808 14:42:58.478310 6960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0ce430212e41a5af7ffd40f1d360ab" candidate_uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c39635bca214f319d45db090ea34993"
I20260808 14:42:58.478379 7092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0ce430212e41a5af7ffd40f1d360ab" candidate_uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c5cb6f8d434f7c9e54c7a364c9852b"
I20260808 14:42:58.478484 6960 raft_consensus.cc:3037] T ae0ce430212e41a5af7ffd40f1d360ab P 0c39635bca214f319d45db090ea34993 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:58.478503 7092 raft_consensus.cc:3037] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:42:58.479529 6960 raft_consensus.cc:2454] T ae0ce430212e41a5af7ffd40f1d360ab P 0c39635bca214f319d45db090ea34993 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fcf906f93dc4f65bf69d2f8ebc65eca in term 1.
I20260808 14:42:58.479565 7092 raft_consensus.cc:2454] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fcf906f93dc4f65bf69d2f8ebc65eca in term 1.
I20260808 14:42:58.479828 7165 leader_election.cc:302] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c39635bca214f319d45db090ea34993, 1fcf906f93dc4f65bf69d2f8ebc65eca; no voters:
I20260808 14:42:58.480012 7304 raft_consensus.cc:2781] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:42:58.480247 7304 raft_consensus.cc:686] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [term 1 LEADER]: Becoming Leader. State: Replica: 1fcf906f93dc4f65bf69d2f8ebc65eca, State: Running, Role: LEADER
I20260808 14:42:58.480585 7304 consensus_queue.cc:237] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39185 } } peers { permanent_uuid: "0c39635bca214f319d45db090ea34993" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 42637 } } peers { permanent_uuid: "d7c5cb6f8d434f7c9e54c7a364c9852b" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44723 } }
I20260808 14:42:58.484018 6814 catalog_manager.cc:5495] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fcf906f93dc4f65bf69d2f8ebc65eca (127.19.144.67). New cstate: current_term: 1 leader_uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fcf906f93dc4f65bf69d2f8ebc65eca" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c39635bca214f319d45db090ea34993" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 42637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7c5cb6f8d434f7c9e54c7a364c9852b" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44723 } health_report { overall_health: UNKNOWN } } }
I20260808 14:42:58.571058 7296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:58.567068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49952 (local address 127.19.144.67:39185)
0808 14:42:58.567186 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 14:42:58.567190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:42:58.567229 (+ 39us) server_negotiation.cc:407] Connection header received
0808 14:42:58.567368 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:42:58.567372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:42:58.567469 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:42:58.567613 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:42:58.567927 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.568617 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.569467 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:42:58.569751 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:42:58.569826 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:42:58.570131 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:42:58.570609 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:42:58.570719 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 14:42:58.570808 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 14:42:58.570842 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 14:42:58.577306 7092 raft_consensus.cc:1261] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b [term 1 FOLLOWER]: Refusing update from remote peer 1fcf906f93dc4f65bf69d2f8ebc65eca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:42:58.577306 6960 raft_consensus.cc:1261] T ae0ce430212e41a5af7ffd40f1d360ab P 0c39635bca214f319d45db090ea34993 [term 1 FOLLOWER]: Refusing update from remote peer 1fcf906f93dc4f65bf69d2f8ebc65eca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:42:58.578136 7308 consensus_queue.cc:1035] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7c5cb6f8d434f7c9e54c7a364c9852b" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:42:58.578346 7304 consensus_queue.cc:1035] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c39635bca214f319d45db090ea34993" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 42637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:42:58.595654 6814 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:42:58.595863 6814 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:42:58.598542 6814 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a97e064a4cf4a21ae4ff9b9ddd6ce29: Sending DeleteTablet for 3 replicas of tablet ae0ce430212e41a5af7ffd40f1d360ab
I20260808 14:42:58.599517 6939 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0ce430212e41a5af7ffd40f1d360ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:42:58 UTC) from {username='slave'} at 127.0.0.1:33130
I20260808 14:42:58.599568 7211 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0ce430212e41a5af7ffd40f1d360ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:42:58 UTC) from {username='slave'} at 127.0.0.1:49942
I20260808 14:42:58.600129 7321 tablet_replica.cc:330] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca: stopping tablet replica
I20260808 14:42:58.600167 7071 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0ce430212e41a5af7ffd40f1d360ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:42:58 UTC) from {username='slave'} at 127.0.0.1:47326
I20260808 14:42:58.600503 7321 raft_consensus.cc:2229] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:42:58.600721 7322 tablet_replica.cc:330] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b: stopping tablet replica
I20260808 14:42:58.600998 7321 raft_consensus.cc:2258] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:42:58.601161 7322 raft_consensus.cc:2229] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:42:58.601263 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 6875
I20260808 14:42:58.601651 7322 raft_consensus.cc:2258] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:42:58.603775 7321 ts_tablet_manager.cc:1905] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:42:58.603998 7322 ts_tablet_manager.cc:1905] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:42:58.606781 7321 ts_tablet_manager.cc:1918] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:42:58.606781 7322 ts_tablet_manager.cc:1918] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:42:58.606901 7321 log.cc:1196] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/ae0ce430212e41a5af7ffd40f1d360ab
I20260808 14:42:58.606901 7322 log.cc:1196] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/ae0ce430212e41a5af7ffd40f1d360ab
I20260808 14:42:58.607260 7322 ts_tablet_manager.cc:1939] T ae0ce430212e41a5af7ffd40f1d360ab P d7c5cb6f8d434f7c9e54c7a364c9852b: Deleting consensus metadata
I20260808 14:42:58.607260 7321 ts_tablet_manager.cc:1939] T ae0ce430212e41a5af7ffd40f1d360ab P 1fcf906f93dc4f65bf69d2f8ebc65eca: Deleting consensus metadata
I20260808 14:42:58.608681 6797 catalog_manager.cc:4841] TS 1fcf906f93dc4f65bf69d2f8ebc65eca (127.19.144.67:39185): tablet ae0ce430212e41a5af7ffd40f1d360ab (table test-table [id=b1c6a77addf24d87892770a5c54b978d]) successfully deleted
I20260808 14:42:58.608696 6799 catalog_manager.cc:4841] TS d7c5cb6f8d434f7c9e54c7a364c9852b (127.19.144.66:44723): tablet ae0ce430212e41a5af7ffd40f1d360ab (table test-table [id=b1c6a77addf24d87892770a5c54b978d]) successfully deleted
W20260808 14:42:58.612859 6801 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.144.65:42637 (error 108)
I20260808 14:42:58.613080 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 7009
I20260808 14:42:58.617973 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:42:58.617540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:42637 (local address 127.0.0.1:33144)
0808 14:42:58.617722 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 14:42:58.617846 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:42637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 14:42:58.618170 6801 catalog_manager.cc:4584] TS 0c39635bca214f319d45db090ea34993 (127.19.144.65:42637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae0ce430212e41a5af7ffd40f1d360ab: Network error: Client connection negotiation failed: client connection to 127.19.144.65:42637: connect: Connection refused (error 111)
I20260808 14:42:58.624145 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 7145
I20260808 14:42:58.631618 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 6782
2026-08-08T14:42:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1460 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:42:58 dist-test-slave-p3t5 krb5kdc[7326](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:42:58 dist-test-slave-p3t5 krb5kdc[7326](info): set up 2 sockets
Aug 08 14:42:58 dist-test-slave-p3t5 krb5kdc[7326](info): commencing operation
krb5kdc: starting...
W20260808 14:42:59.699182 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:42:59 dist-test-slave-p3t5 krb5kdc[7326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:42:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:42:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:42:59.841356 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:34841
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:33055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:34841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:42:59.963148 7340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:42:59.963491 7340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:42:59.963536 7340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:42:59.967764 7340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:42:59.967849 7340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:42:59.967869 7340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:42:59.967887 7340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:42:59.967902 7340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:42:59.973695 7340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:33055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:34841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:34841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:42:59.974897 7340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:42:59.975970 7340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:42:59.985390 7348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:42:59.985394 7345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:59.985529 7340 server_base.cc:1034] running on GCE node
W20260808 14:42:59.985420 7346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:42:59.986493 7340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:42:59.987775 7340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:42:59.988986 7340 hybrid_clock.cc:648] HybridClock initialized: now 1786200179988970 us; error 35 us; skew 500 ppm
Aug 08 14:42:59 dist-test-slave-p3t5 krb5kdc[7326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200179, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:42:59.992383 7340 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:42:59.993448 7340 webserver.cc:460] Webserver started at http://127.19.144.126:44735/ using document root <none> and password file <none>
I20260808 14:42:59.994117 7340 fs_manager.cc:359] Metadata directory not provided
I20260808 14:42:59.994185 7340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:42:59.994454 7340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:42:59.996682 7340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "d24ce6efb5f0486199001fd7a19506ad"
format_stamp: "Formatted at 2026-08-08 14:42:59 on dist-test-slave-p3t5"
server_key: "3e84c18ac200c1fd44e76a581056266a"
server_key_iv: "5274529954acf0ea04f71d17a2e24073"
server_key_version: "encryptionkey@0"
I20260808 14:42:59.997296 7340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "d24ce6efb5f0486199001fd7a19506ad"
format_stamp: "Formatted at 2026-08-08 14:42:59 on dist-test-slave-p3t5"
server_key: "3e84c18ac200c1fd44e76a581056266a"
server_key_iv: "5274529954acf0ea04f71d17a2e24073"
server_key_version: "encryptionkey@0"
I20260808 14:43:00.001462 7340 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 14:43:00.004338 7355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:00.005749 7340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:43:00.005905 7340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "d24ce6efb5f0486199001fd7a19506ad"
format_stamp: "Formatted at 2026-08-08 14:42:59 on dist-test-slave-p3t5"
server_key: "3e84c18ac200c1fd44e76a581056266a"
server_key_iv: "5274529954acf0ea04f71d17a2e24073"
server_key_version: "encryptionkey@0"
I20260808 14:43:00.006039 7340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:00.042325 7340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:00.045936 7340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:00.046162 7340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:00.053977 7340 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:34841
I20260808 14:43:00.054004 7409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:34841 every 8 connection(s)
I20260808 14:43:00.055058 7340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:00.058166 7410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:00.059926 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 7340
I20260808 14:43:00.060084 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:00.060424 20033 external_mini_cluster.cc:1428] Setting key 14aeeba0e82aebd76ecd40723a7c0c40
I20260808 14:43:00.065008 7410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad: Bootstrap starting.
I20260808 14:43:00.067492 7410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:00.068306 7410 log.cc:824] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:43:00 dist-test-slave-p3t5 krb5kdc[7326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:43:00.070562 7410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad: No bootstrap required, opened a new log
I20260808 14:43:00.074132 7410 raft_consensus.cc:348] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24ce6efb5f0486199001fd7a19506ad" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34841 } }
I20260808 14:43:00.074404 7410 raft_consensus.cc:374] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:00.074463 7410 raft_consensus.cc:729] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d24ce6efb5f0486199001fd7a19506ad, State: Initialized, Role: FOLLOWER
I20260808 14:43:00.074903 7410 consensus_queue.cc:260] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24ce6efb5f0486199001fd7a19506ad" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34841 } }
I20260808 14:43:00.075067 7410 raft_consensus.cc:388] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:00.075147 7410 raft_consensus.cc:482] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:00.075268 7410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:00.075510 7413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:00.062061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32780 (local address 127.19.144.126:34841)
0808 14:43:00.062575 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:00.062593 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:00.062631 (+ 38us) server_negotiation.cc:407] Connection header received
0808 14:43:00.063282 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:00.063299 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:00.063655 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:00.064013 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:00.064896 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.066033 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.066927 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.067447 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.067627 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:00.070468 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:00.070474 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:00.070488 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:00.070517 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:00.072321 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:00.072791 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:00.072796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:00.072801 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:00.072919 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:00.073215 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:00.073218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:00.073220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:00.073708 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:00.073936 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:00.074208 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 14:43:00.074517 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":163,"threads_started":1}
I20260808 14:43:00.076507 7410 raft_consensus.cc:504] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24ce6efb5f0486199001fd7a19506ad" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34841 } }
I20260808 14:43:00.076870 7410 leader_election.cc:302] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d24ce6efb5f0486199001fd7a19506ad; no voters:
I20260808 14:43:00.077230 7410 leader_election.cc:288] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:00.077350 7415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:00.077534 7415 raft_consensus.cc:686] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [term 1 LEADER]: Becoming Leader. State: Replica: d24ce6efb5f0486199001fd7a19506ad, State: Running, Role: LEADER
I20260808 14:43:00.077864 7415 consensus_queue.cc:237] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24ce6efb5f0486199001fd7a19506ad" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34841 } }
I20260808 14:43:00.078603 7410 sys_catalog.cc:564] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:00.079115 7415 sys_catalog.cc:455] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader d24ce6efb5f0486199001fd7a19506ad. Latest consensus state: current_term: 1 leader_uuid: "d24ce6efb5f0486199001fd7a19506ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24ce6efb5f0486199001fd7a19506ad" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34841 } } }
I20260808 14:43:00.079257 7415 sys_catalog.cc:458] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:00.079574 7416 sys_catalog.cc:455] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d24ce6efb5f0486199001fd7a19506ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24ce6efb5f0486199001fd7a19506ad" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34841 } } }
I20260808 14:43:00.079686 7416 sys_catalog.cc:458] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:00.079705 7419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:00.083194 7419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:00.089206 7419 catalog_manager.cc:1347] Generated new cluster ID: 2820ac2d10ca4e019c3b3378768c7195
I20260808 14:43:00.089304 7419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:00.097843 7419 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:00.099212 7419 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:00.116178 7419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad: Generated new TSK 0
I20260808 14:43:00.117154 7419 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.144.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
Entry for principal kudu/127.19.144.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab.
I20260808 14:43:00.161968 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34841
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:33055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:00.282948 7437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:00.283257 7437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:00.283303 7437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:00.287489 7437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:00.287560 7437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:00.287621 7437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:43:00.293114 7437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:33055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.65.keytab
--principal=kudu/127.19.144.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:00.294320 7437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:00.295346 7437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:43:00.305754 7437 server_base.cc:1034] running on GCE node
W20260808 14:43:00.305752 7445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:00.305745 7442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:00.306005 7443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:00.306474 7437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:00.307171 7437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:00.308344 7437 hybrid_clock.cc:648] HybridClock initialized: now 1786200180308326 us; error 30 us; skew 500 ppm
Aug 08 14:43:00 dist-test-slave-p3t5 krb5kdc[7326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200180, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:00.311739 7437 init.cc:377] Logged in from keytab as kudu/127.19.144.65@KRBTEST.COM (short username kudu)
I20260808 14:43:00.312745 7437 webserver.cc:460] Webserver started at http://127.19.144.65:33401/ using document root <none> and password file <none>
I20260808 14:43:00.313361 7437 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:00.313429 7437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:00.313709 7437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:00.315937 7437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "ca031c871ee44f7d8708b131866c267d"
format_stamp: "Formatted at 2026-08-08 14:43:00 on dist-test-slave-p3t5"
server_key: "a8afe8ab0f371b637dd1adb3805472b7"
server_key_iv: "361d6c1074ad8fb6c307b5955a60429d"
server_key_version: "encryptionkey@0"
I20260808 14:43:00.316542 7437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "ca031c871ee44f7d8708b131866c267d"
format_stamp: "Formatted at 2026-08-08 14:43:00 on dist-test-slave-p3t5"
server_key: "a8afe8ab0f371b637dd1adb3805472b7"
server_key_iv: "361d6c1074ad8fb6c307b5955a60429d"
server_key_version: "encryptionkey@0"
I20260808 14:43:00.320575 7437 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:43:00.323251 7452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:00.324532 7437 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 14:43:00.324680 7437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "ca031c871ee44f7d8708b131866c267d"
format_stamp: "Formatted at 2026-08-08 14:43:00 on dist-test-slave-p3t5"
server_key: "a8afe8ab0f371b637dd1adb3805472b7"
server_key_iv: "361d6c1074ad8fb6c307b5955a60429d"
server_key_version: "encryptionkey@0"
I20260808 14:43:00.324795 7437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:00.339684 7437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:00.342864 7437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:00.343027 7437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:00.343613 7437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:43:00.344437 7437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:43:00.344487 7437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:00.344528 7437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:43:00.344550 7437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:00.353457 7437 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:40423
I20260808 14:43:00.353482 7566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:40423 every 8 connection(s)
I20260808 14:43:00.354476 7437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:43:00.360451 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 7437
I20260808 14:43:00.360598 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:43:00.360925 20033 external_mini_cluster.cc:1428] Setting key 8285c281251d314957fb8799aa7e589d
Aug 08 14:43:00 dist-test-slave-p3t5 krb5kdc[7326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200180, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.65@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:43:00.371279 7413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:00.356398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:42561 (local address 127.19.144.126:34841)
0808 14:43:00.356567 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:00.356572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:00.357497 (+ 925us) server_negotiation.cc:407] Connection header received
0808 14:43:00.359015 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:00.359018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:00.359073 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:00.359158 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:00.360781 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.361455 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.363078 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.363322 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.363415 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:00.366936 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:00.366940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:00.366942 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:00.366969 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:00.368674 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:00.369415 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:00.369417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:00.369419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:00.369501 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:00.369982 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:00.369985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:00.369986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:00.370186 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:00.370269 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:00.371027 (+ 758us) server_negotiation.cc:299] Negotiation successful
0808 14:43:00.371147 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 14:43:00.372663 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:00.356861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:34841 (local address 127.19.144.65:42561)
0808 14:43:00.357337 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:00.357383 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:00.358696 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:00.359420 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:00.359428 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:00.359858 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:00.360582 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:00.360608 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.361632 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.361636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:00.362922 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:00.362935 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.363498 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.363505 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:00.363776 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:00.364486 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:00.364515 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:00.366704 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:00.368814 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:00.368822 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:00.368843 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:00.369294 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:00.369598 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:00.369604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:00.369608 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:00.369869 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:00.370302 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:00.370312 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:00.370735 (+ 423us) client_negotiation.cc:769] Sending connection context
0808 14:43:00.370987 (+ 252us) client_negotiation.cc:240] Negotiation successful
0808 14:43:00.371414 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.19.144.66@KRBTEST.COM; defaulting to no policy
I20260808 14:43:00.374084 7567 heartbeater.cc:347] Connected to a master server at 127.19.144.126:34841
I20260808 14:43:00.374358 7567 heartbeater.cc:464] Registering TS with master...
I20260808 14:43:00.374907 7567 heartbeater.cc:511] Master 127.19.144.126:34841 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.66@KRBTEST.COM" created.
I20260808 14:43:00.376750 7372 ts_manager.cc:194] Registered new tserver with Master: ca031c871ee44f7d8708b131866c267d (127.19.144.65:40423)
I20260808 14:43:00.378022 7372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.65@KRBTEST.COM'} at 127.19.144.65:42561
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
Entry for principal kudu/127.19.144.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab.
I20260808 14:43:00.396764 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34841
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:33055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:00.516399 7572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:00.516680 7572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:00.516724 7572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:00.520784 7572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:00.520867 7572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:00.520927 7572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:43:00.526583 7572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:33055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.66.keytab
--principal=kudu/127.19.144.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:00.527740 7572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:00.528859 7572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:00.539062 7578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:00.539122 7579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:00.539113 7581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:00.539361 7572 server_base.cc:1034] running on GCE node
I20260808 14:43:00.539803 7572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:00.540488 7572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:00.541677 7572 hybrid_clock.cc:648] HybridClock initialized: now 1786200180541614 us; error 72 us; skew 500 ppm
Aug 08 14:43:00 dist-test-slave-p3t5 krb5kdc[7326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200180, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:00.544605 7572 init.cc:377] Logged in from keytab as kudu/127.19.144.66@KRBTEST.COM (short username kudu)
I20260808 14:43:00.545579 7572 webserver.cc:460] Webserver started at http://127.19.144.66:36685/ using document root <none> and password file <none>
I20260808 14:43:00.546224 7572 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:00.546329 7572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:00.546571 7572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:00.548867 7572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "d798462d34a7443e9382fc5c43ff8b17"
format_stamp: "Formatted at 2026-08-08 14:43:00 on dist-test-slave-p3t5"
server_key: "1ccc9c6799c9d879ad8955ba05949016"
server_key_iv: "42a2ba706c07e106eaf5d6a642e7827b"
server_key_version: "encryptionkey@0"
I20260808 14:43:00.549469 7572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "d798462d34a7443e9382fc5c43ff8b17"
format_stamp: "Formatted at 2026-08-08 14:43:00 on dist-test-slave-p3t5"
server_key: "1ccc9c6799c9d879ad8955ba05949016"
server_key_iv: "42a2ba706c07e106eaf5d6a642e7827b"
server_key_version: "encryptionkey@0"
I20260808 14:43:00.553481 7572 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260808 14:43:00.556249 7588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:00.557866 7572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:43:00.558014 7572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "d798462d34a7443e9382fc5c43ff8b17"
format_stamp: "Formatted at 2026-08-08 14:43:00 on dist-test-slave-p3t5"
server_key: "1ccc9c6799c9d879ad8955ba05949016"
server_key_iv: "42a2ba706c07e106eaf5d6a642e7827b"
server_key_version: "encryptionkey@0"
I20260808 14:43:00.558168 7572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:00.572203 7572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:00.575881 7572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:00.576093 7572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:00.576728 7572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:43:00.577705 7572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:43:00.577755 7572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:00.577829 7572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:43:00.577862 7572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:00.587476 7572 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:46805
I20260808 14:43:00.587481 7704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:46805 every 8 connection(s)
I20260808 14:43:00.588483 7572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:43:00.595227 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 7572
I20260808 14:43:00.595336 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:43:00.595629 20033 external_mini_cluster.cc:1428] Setting key 36e6b64db3e3f25387a37f902fbeba3c
Aug 08 14:43:00 dist-test-slave-p3t5 krb5kdc[7326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200180, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.66@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:43:00.604372 7413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:00.590394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:58645 (local address 127.19.144.126:34841)
0808 14:43:00.590542 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:00.590546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:00.591863 (+ 1317us) server_negotiation.cc:407] Connection header received
0808 14:43:00.592811 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:00.592815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:00.592876 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:00.593004 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:00.594354 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.594906 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.596458 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.596687 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.596799 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:00.600049 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:00.600052 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:00.600054 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:00.600081 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:00.601896 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:00.602479 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:00.602485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:00.602490 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:00.602609 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:00.602996 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:00.603002 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:00.603007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:00.603292 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:00.603406 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:00.603951 (+ 545us) server_negotiation.cc:299] Negotiation successful
0808 14:43:00.604115 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:43:00.605181 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:00.590804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:34841 (local address 127.19.144.66:58645)
0808 14:43:00.591702 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:00.591749 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:00.592590 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:00.593195 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:00.593202 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:00.593597 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:00.594207 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:00.594225 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.595169 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.595172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:00.596322 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:00.596333 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.596843 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.596848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:00.597029 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:00.597736 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:00.597761 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:00.599849 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:00.602022 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:00.602028 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:00.602042 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:00.602351 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:00.602739 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:00.602742 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:00.602743 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:00.602901 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:00.603403 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:00.603409 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:00.603682 (+ 273us) client_negotiation.cc:769] Sending connection context
0808 14:43:00.603850 (+ 168us) client_negotiation.cc:240] Negotiation successful
0808 14:43:00.604136 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":559,"threads_started":1}
I20260808 14:43:00.606561 7705 heartbeater.cc:347] Connected to a master server at 127.19.144.126:34841
I20260808 14:43:00.606839 7705 heartbeater.cc:464] Registering TS with master...
I20260808 14:43:00.607383 7705 heartbeater.cc:511] Master 127.19.144.126:34841 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.144.67@KRBTEST.COM; defaulting to no policy
I20260808 14:43:00.608552 7372 ts_manager.cc:194] Registered new tserver with Master: d798462d34a7443e9382fc5c43ff8b17 (127.19.144.66:46805)
I20260808 14:43:00.609164 7372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.66@KRBTEST.COM'} at 127.19.144.66:58645
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
Entry for principal kudu/127.19.144.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab.
I20260808 14:43:00.629125 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34841
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:33055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:00.745338 7710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:00.745709 7710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:00.745790 7710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:00.750383 7710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:00.750496 7710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:00.750599 7710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:43:00.756165 7710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:33055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.67.keytab
--principal=kudu/127.19.144.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:34841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:00.757421 7710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:00.758613 7710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:00.768994 7715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:00.769043 7719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:00.769024 7716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:00.769304 7710 server_base.cc:1034] running on GCE node
I20260808 14:43:00.769840 7710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:00.770561 7710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:00.771728 7710 hybrid_clock.cc:648] HybridClock initialized: now 1786200180771707 us; error 31 us; skew 500 ppm
Aug 08 14:43:00 dist-test-slave-p3t5 krb5kdc[7326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200180, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:00.774842 7710 init.cc:377] Logged in from keytab as kudu/127.19.144.67@KRBTEST.COM (short username kudu)
I20260808 14:43:00.775759 7710 webserver.cc:460] Webserver started at http://127.19.144.67:34235/ using document root <none> and password file <none>
I20260808 14:43:00.776398 7710 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:00.776495 7710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:00.776738 7710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:00.779140 7710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "ec6cb6a1afde494ba8642f4db74aa0b2"
format_stamp: "Formatted at 2026-08-08 14:43:00 on dist-test-slave-p3t5"
server_key: "2cb6a6a5dba83059592d8c27670215d8"
server_key_iv: "355b1150fd9338ecb87897382acca661"
server_key_version: "encryptionkey@0"
I20260808 14:43:00.779883 7710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "ec6cb6a1afde494ba8642f4db74aa0b2"
format_stamp: "Formatted at 2026-08-08 14:43:00 on dist-test-slave-p3t5"
server_key: "2cb6a6a5dba83059592d8c27670215d8"
server_key_iv: "355b1150fd9338ecb87897382acca661"
server_key_version: "encryptionkey@0"
I20260808 14:43:00.784714 7710 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 14:43:00.787647 7726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:00.789055 7710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:43:00.789214 7710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "ec6cb6a1afde494ba8642f4db74aa0b2"
format_stamp: "Formatted at 2026-08-08 14:43:00 on dist-test-slave-p3t5"
server_key: "2cb6a6a5dba83059592d8c27670215d8"
server_key_iv: "355b1150fd9338ecb87897382acca661"
server_key_version: "encryptionkey@0"
I20260808 14:43:00.789350 7710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:00.817830 7710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:00.821128 7710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:00.821310 7710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:00.821920 7710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:43:00.822811 7710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:43:00.822865 7710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:00.822908 7710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:43:00.822923 7710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:00.832353 7710 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:45935
I20260808 14:43:00.832413 7843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:45935 every 8 connection(s)
I20260808 14:43:00.833377 7710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:43:00.837082 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 7710
I20260808 14:43:00.837224 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:43:00.837513 20033 external_mini_cluster.cc:1428] Setting key 069c8c8ff1821a737307a60d4d283ff2
Aug 08 14:43:00 dist-test-slave-p3t5 krb5kdc[7326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200180, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.67@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:43:00.847885 7413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:00.835345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:59695 (local address 127.19.144.126:34841)
0808 14:43:00.835503 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:00.835507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:00.836338 (+ 831us) server_negotiation.cc:407] Connection header received
0808 14:43:00.837984 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:00.837988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:00.838053 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:00.838157 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:00.839409 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.840062 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.841307 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.841543 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.841682 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:00.844292 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:00.844296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:00.844304 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:00.844332 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:00.845695 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:00.846240 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:00.846244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:00.846246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:00.846333 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:00.846712 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:00.846725 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:00.846726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:00.846929 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:00.847042 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:00.847570 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 14:43:00.847710 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 14:43:00.848775 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:00.835708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:34841 (local address 127.19.144.67:59695)
0808 14:43:00.836173 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:00.836220 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:00.837746 (+ 1526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:00.838348 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:00.838355 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:00.838731 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:00.839248 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:00.839262 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.840195 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.840198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:00.841164 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:00.841180 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.841719 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.841723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:00.841926 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:00.842512 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:00.842529 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:00.844114 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:00.845815 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:00.845821 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:00.845840 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:00.846128 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:00.846451 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:00.846454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:00.846456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:00.846610 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:00.847039 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:00.847046 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:00.847321 (+ 275us) client_negotiation.cc:769] Sending connection context
0808 14:43:00.847525 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 14:43:00.847889 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":124,"threads_started":1}
I20260808 14:43:00.850068 7844 heartbeater.cc:347] Connected to a master server at 127.19.144.126:34841
I20260808 14:43:00.850363 7844 heartbeater.cc:464] Registering TS with master...
I20260808 14:43:00.851038 7844 heartbeater.cc:511] Master 127.19.144.126:34841 requested a full tablet report, sending...
I20260808 14:43:00.852166 7372 ts_manager.cc:194] Registered new tserver with Master: ec6cb6a1afde494ba8642f4db74aa0b2 (127.19.144.67:45935)
I20260808 14:43:00.852794 7372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.144.67@KRBTEST.COM'} at 127.19.144.67:59695
I20260808 14:43:00.863729 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:43:00.873160 7413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:00.865962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32788 (local address 127.19.144.126:34841)
0808 14:43:00.866116 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:00.866121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:00.866325 (+ 204us) server_negotiation.cc:407] Connection header received
0808 14:43:00.866442 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:00.866445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:00.866493 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:00.866553 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:00.867257 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.867777 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.868577 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.868782 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.868858 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:00.870314 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:00.870317 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:00.870319 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:00.870340 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:00.871612 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:00.872003 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:00.872005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:00.872006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:00.872085 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:00.872307 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:00.872309 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:00.872311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:00.872539 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:00.872644 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:00.872842 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 14:43:00.872950 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:43:00.876317 7372 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:43:00.879702 7372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:43:00.895159 7858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:00.889695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:40423 (local address 127.0.0.1:58242)
0808 14:43:00.890051 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:00.890066 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:00.890153 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:00.891061 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:00.891064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:00.891082 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:43:00.891203 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:00.891212 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.892364 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.892368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:00.893440 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:00.893453 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.894588 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.894592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:00.894696 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:00.894726 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 14:43:00.894816 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 14:43:00.894999 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":4,"thread_start_us":101,"threads_started":1}
I20260808 14:43:00.895160 7856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:00.889029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.19.144.67:45935)
0808 14:43:00.889447 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:00.889460 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:00.889489 (+ 29us) server_negotiation.cc:407] Connection header received
0808 14:43:00.889676 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:00.889683 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:00.889924 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:00.890103 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:43:00.890642 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.891763 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.893644 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.894355 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.894481 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:00.894715 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:00.894899 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 14:43:00.895009 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":180,"threads_started":1}
I20260808 14:43:00.895537 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:00.890135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58242 (local address 127.19.144.65:40423)
0808 14:43:00.890613 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:00.890620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:00.890644 (+ 24us) server_negotiation.cc:407] Connection header received
0808 14:43:00.890724 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:00.890729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:00.890918 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:00.891075 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:43:00.891337 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.892236 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.893584 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.894388 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.895094 (+ 706us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:00.895230 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:00.895280 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 14:43:00.895387 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":94,"threads_started":1}
I20260808 14:43:00.895551 7857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:00.889703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:46805 (local address 127.0.0.1:56650)
0808 14:43:00.890500 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:00.890515 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:00.890602 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:00.891223 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:00.891226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:00.891244 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:43:00.891348 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:00.891354 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.892489 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.892493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:00.893876 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:00.893887 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.895165 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.895168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:00.895257 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:00.895279 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 14:43:00.895351 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 14:43:00.895413 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":127,"thread_start_us":115,"threads_started":1}
I20260808 14:43:00.895676 7855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:00.888883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:45935 (local address 127.0.0.1:51458)
0808 14:43:00.889262 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:00.889321 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:00.889472 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:00.890312 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:00.890319 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:00.890344 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:43:00.890492 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:00.890500 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.891899 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.891902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:00.893473 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:00.893486 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.894482 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.894485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:00.894658 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:00.894717 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 14:43:00.894849 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 14:43:00.894997 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":124,"thread_start_us":90,"threads_started":1}
I20260808 14:43:00.898015 7859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:00.889877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56650 (local address 127.19.144.66:46805)
0808 14:43:00.890406 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:00.890418 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:00.890701 (+ 283us) server_negotiation.cc:407] Connection header received
0808 14:43:00.890786 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:00.890792 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:00.890998 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:00.891146 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:43:00.891469 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.892356 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.894005 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:00.894852 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:00.894954 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:00.895100 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:00.897793 (+ 2693us) server_negotiation.cc:299] Negotiation successful
0808 14:43:00.897892 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":100,"threads_started":1}
I20260808 14:43:00.899077 7501 tablet_service.cc:1467] Processing CreateTablet for tablet 982ecb92cffb4c1ea916b2517b18c4c4 (DEFAULT_TABLE table=test-table [id=c7752fcceefb4b32bc1ff65760f61914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:43:00.899080 7777 tablet_service.cc:1467] Processing CreateTablet for tablet 982ecb92cffb4c1ea916b2517b18c4c4 (DEFAULT_TABLE table=test-table [id=c7752fcceefb4b32bc1ff65760f61914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:43:00.900336 7501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 982ecb92cffb4c1ea916b2517b18c4c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:00.900336 7777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 982ecb92cffb4c1ea916b2517b18c4c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:00.900776 7637 tablet_service.cc:1467] Processing CreateTablet for tablet 982ecb92cffb4c1ea916b2517b18c4c4 (DEFAULT_TABLE table=test-table [id=c7752fcceefb4b32bc1ff65760f61914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:43:00.901813 7637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 982ecb92cffb4c1ea916b2517b18c4c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:00.908181 7862 tablet_bootstrap.cc:492] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d: Bootstrap starting.
I20260808 14:43:00.908684 7863 tablet_bootstrap.cc:492] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2: Bootstrap starting.
I20260808 14:43:00.909600 7864 tablet_bootstrap.cc:492] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17: Bootstrap starting.
I20260808 14:43:00.911375 7863 tablet_bootstrap.cc:654] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:00.912024 7862 tablet_bootstrap.cc:654] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:00.912043 7864 tablet_bootstrap.cc:654] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:00.912789 7862 log.cc:824] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:00.912927 7863 log.cc:824] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:00.913327 7864 log.cc:824] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:00.915203 7862 tablet_bootstrap.cc:492] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d: No bootstrap required, opened a new log
I20260808 14:43:00.915444 7862 ts_tablet_manager.cc:1397] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 14:43:00.915498 7864 tablet_bootstrap.cc:492] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17: No bootstrap required, opened a new log
I20260808 14:43:00.915665 7864 ts_tablet_manager.cc:1397] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260808 14:43:00.918838 7863 tablet_bootstrap.cc:492] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2: No bootstrap required, opened a new log
I20260808 14:43:00.919081 7863 ts_tablet_manager.cc:1397] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260808 14:43:00.919831 7864 raft_consensus.cc:348] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6cb6a1afde494ba8642f4db74aa0b2" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 45935 } } peers { permanent_uuid: "ca031c871ee44f7d8708b131866c267d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 40423 } } peers { permanent_uuid: "d798462d34a7443e9382fc5c43ff8b17" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46805 } }
I20260808 14:43:00.920154 7864 raft_consensus.cc:374] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:00.920224 7864 raft_consensus.cc:729] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d798462d34a7443e9382fc5c43ff8b17, State: Initialized, Role: FOLLOWER
I20260808 14:43:00.920044 7862 raft_consensus.cc:348] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6cb6a1afde494ba8642f4db74aa0b2" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 45935 } } peers { permanent_uuid: "ca031c871ee44f7d8708b131866c267d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 40423 } } peers { permanent_uuid: "d798462d34a7443e9382fc5c43ff8b17" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46805 } }
I20260808 14:43:00.920320 7862 raft_consensus.cc:374] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:00.920387 7862 raft_consensus.cc:729] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca031c871ee44f7d8708b131866c267d, State: Initialized, Role: FOLLOWER
I20260808 14:43:00.920794 7862 consensus_queue.cc:260] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6cb6a1afde494ba8642f4db74aa0b2" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 45935 } } peers { permanent_uuid: "ca031c871ee44f7d8708b131866c267d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 40423 } } peers { permanent_uuid: "d798462d34a7443e9382fc5c43ff8b17" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46805 } }
I20260808 14:43:00.920795 7864 consensus_queue.cc:260] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6cb6a1afde494ba8642f4db74aa0b2" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 45935 } } peers { permanent_uuid: "ca031c871ee44f7d8708b131866c267d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 40423 } } peers { permanent_uuid: "d798462d34a7443e9382fc5c43ff8b17" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46805 } }
I20260808 14:43:00.921854 7705 heartbeater.cc:503] Master 127.19.144.126:34841 was elected leader, sending a full tablet report...
I20260808 14:43:00.921875 7567 heartbeater.cc:503] Master 127.19.144.126:34841 was elected leader, sending a full tablet report...
I20260808 14:43:00.921962 7864 ts_tablet_manager.cc:1428] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260808 14:43:00.923082 7862 ts_tablet_manager.cc:1428] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d: Time spent starting tablet: real 0.008s user 0.001s sys 0.007s
I20260808 14:43:00.922915 7863 raft_consensus.cc:348] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6cb6a1afde494ba8642f4db74aa0b2" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 45935 } } peers { permanent_uuid: "ca031c871ee44f7d8708b131866c267d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 40423 } } peers { permanent_uuid: "d798462d34a7443e9382fc5c43ff8b17" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46805 } }
I20260808 14:43:00.923233 7863 raft_consensus.cc:374] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:00.923292 7863 raft_consensus.cc:729] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec6cb6a1afde494ba8642f4db74aa0b2, State: Initialized, Role: FOLLOWER
I20260808 14:43:00.923784 7863 consensus_queue.cc:260] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6cb6a1afde494ba8642f4db74aa0b2" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 45935 } } peers { permanent_uuid: "ca031c871ee44f7d8708b131866c267d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 40423 } } peers { permanent_uuid: "d798462d34a7443e9382fc5c43ff8b17" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46805 } }
I20260808 14:43:00.924691 7844 heartbeater.cc:503] Master 127.19.144.126:34841 was elected leader, sending a full tablet report...
I20260808 14:43:00.924887 7863 ts_tablet_manager.cc:1428] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:43:01.084364 7870 raft_consensus.cc:482] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 14:43:01.084424 7845 tablet.cc:2369] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:43:01.084633 7870 raft_consensus.cc:504] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6cb6a1afde494ba8642f4db74aa0b2" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 45935 } } peers { permanent_uuid: "ca031c871ee44f7d8708b131866c267d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 40423 } } peers { permanent_uuid: "d798462d34a7443e9382fc5c43ff8b17" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46805 } }
I20260808 14:43:01.085875 7870 leader_election.cc:288] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca031c871ee44f7d8708b131866c267d (127.19.144.65:40423), d798462d34a7443e9382fc5c43ff8b17 (127.19.144.66:46805)
W20260808 14:43:01.089696 7706 tablet.cc:2369] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:43:01.089793 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:01.086085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:40423 (local address 127.19.144.67:50063)
0808 14:43:01.086228 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:01.086247 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:01.086348 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:01.086690 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:01.086694 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:01.086743 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:43:01.086878 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:01.086888 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.087712 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.087716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:01.088570 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:01.088580 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.089295 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.089298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:01.089447 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:01.089465 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 14:43:01.089553 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 14:43:01.089646 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":2}
I20260808 14:43:01.089792 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:01.086293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:50063 (local address 127.19.144.65:40423)
0808 14:43:01.086429 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:01.086433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:01.086451 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:43:01.086513 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:01.086516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:01.086568 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:01.086663 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:43:01.087008 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.087600 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.088721 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.089166 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.089299 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:01.089354 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:01.089577 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 14:43:01.089681 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 14:43:01.089893 7859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:01.086328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:47751 (local address 127.19.144.66:46805)
0808 14:43:01.086532 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:01.086536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:01.086661 (+ 125us) server_negotiation.cc:407] Connection header received
0808 14:43:01.086782 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:01.086785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:01.086843 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:01.086954 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:43:01.087228 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.087741 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.088784 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.089174 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.089300 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:01.089354 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:01.089699 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 14:43:01.089785 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260808 14:43:01.090224 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:01.086162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:46805 (local address 127.19.144.67:47751)
0808 14:43:01.086566 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:01.086581 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:01.086674 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:01.086971 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:01.086974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:01.086993 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:43:01.087116 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:01.087123 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.087852 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.087855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:01.088617 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:01.088656 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.089307 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.089311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:01.089510 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:01.089534 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 14:43:01.090021 (+ 487us) client_negotiation.cc:240] Negotiation successful
0808 14:43:01.090095 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":71,"thread_start_us":136,"threads_started":1}
I20260808 14:43:01.090765 7521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "982ecb92cffb4c1ea916b2517b18c4c4" candidate_uuid: "ec6cb6a1afde494ba8642f4db74aa0b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca031c871ee44f7d8708b131866c267d" is_pre_election: true
I20260808 14:43:01.090837 7658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "982ecb92cffb4c1ea916b2517b18c4c4" candidate_uuid: "ec6cb6a1afde494ba8642f4db74aa0b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d798462d34a7443e9382fc5c43ff8b17" is_pre_election: true
I20260808 14:43:01.091115 7521 raft_consensus.cc:2454] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec6cb6a1afde494ba8642f4db74aa0b2 in term 0.
I20260808 14:43:01.091128 7658 raft_consensus.cc:2454] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec6cb6a1afde494ba8642f4db74aa0b2 in term 0.
I20260808 14:43:01.091642 7727 leader_election.cc:302] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: ca031c871ee44f7d8708b131866c267d, d798462d34a7443e9382fc5c43ff8b17, ec6cb6a1afde494ba8642f4db74aa0b2; no voters:
I20260808 14:43:01.091959 7870 raft_consensus.cc:2781] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:43:01.092051 7870 raft_consensus.cc:482] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:43:01.092095 7870 raft_consensus.cc:3037] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:01.093179 7870 raft_consensus.cc:504] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6cb6a1afde494ba8642f4db74aa0b2" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 45935 } } peers { permanent_uuid: "ca031c871ee44f7d8708b131866c267d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 40423 } } peers { permanent_uuid: "d798462d34a7443e9382fc5c43ff8b17" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46805 } }
I20260808 14:43:01.093673 7870 leader_election.cc:288] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [CANDIDATE]: Term 1 election: Requested vote from peers ca031c871ee44f7d8708b131866c267d (127.19.144.65:40423), d798462d34a7443e9382fc5c43ff8b17 (127.19.144.66:46805)
I20260808 14:43:01.094074 7521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "982ecb92cffb4c1ea916b2517b18c4c4" candidate_uuid: "ec6cb6a1afde494ba8642f4db74aa0b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca031c871ee44f7d8708b131866c267d"
I20260808 14:43:01.094090 7658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "982ecb92cffb4c1ea916b2517b18c4c4" candidate_uuid: "ec6cb6a1afde494ba8642f4db74aa0b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d798462d34a7443e9382fc5c43ff8b17"
I20260808 14:43:01.094188 7521 raft_consensus.cc:3037] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:01.094224 7658 raft_consensus.cc:3037] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:01.095310 7521 raft_consensus.cc:2454] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec6cb6a1afde494ba8642f4db74aa0b2 in term 1.
I20260808 14:43:01.095359 7658 raft_consensus.cc:2454] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec6cb6a1afde494ba8642f4db74aa0b2 in term 1.
I20260808 14:43:01.095723 7727 leader_election.cc:302] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d798462d34a7443e9382fc5c43ff8b17, ec6cb6a1afde494ba8642f4db74aa0b2; no voters:
I20260808 14:43:01.095933 7870 raft_consensus.cc:2781] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:01.096194 7870 raft_consensus.cc:686] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [term 1 LEADER]: Becoming Leader. State: Replica: ec6cb6a1afde494ba8642f4db74aa0b2, State: Running, Role: LEADER
I20260808 14:43:01.096551 7870 consensus_queue.cc:237] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6cb6a1afde494ba8642f4db74aa0b2" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 45935 } } peers { permanent_uuid: "ca031c871ee44f7d8708b131866c267d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 40423 } } peers { permanent_uuid: "d798462d34a7443e9382fc5c43ff8b17" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46805 } }
I20260808 14:43:01.099867 7366 catalog_manager.cc:5495] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec6cb6a1afde494ba8642f4db74aa0b2 (127.19.144.67). New cstate: current_term: 1 leader_uuid: "ec6cb6a1afde494ba8642f4db74aa0b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6cb6a1afde494ba8642f4db74aa0b2" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 45935 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca031c871ee44f7d8708b131866c267d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 40423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d798462d34a7443e9382fc5c43ff8b17" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46805 } health_report { overall_health: UNKNOWN } } }
W20260808 14:43:01.105808 7568 tablet.cc:2369] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:43:01.137982 7856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:01.134166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51462 (local address 127.19.144.67:45935)
0808 14:43:01.134310 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:01.134315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:01.134331 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:43:01.134494 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:01.134496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:01.134557 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:01.134675 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:43:01.134952 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.135477 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.136360 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.136600 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.136714 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:01.137051 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:43:01.137522 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:43:01.137691 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:01.137733 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 14:43:01.137797 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 14:43:01.144981 7658 raft_consensus.cc:1261] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17 [term 1 FOLLOWER]: Refusing update from remote peer ec6cb6a1afde494ba8642f4db74aa0b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:43:01.144981 7521 raft_consensus.cc:1261] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d [term 1 FOLLOWER]: Refusing update from remote peer ec6cb6a1afde494ba8642f4db74aa0b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:43:01.145821 7872 consensus_queue.cc:1035] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca031c871ee44f7d8708b131866c267d" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 40423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:43:01.146051 7870 consensus_queue.cc:1035] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d798462d34a7443e9382fc5c43ff8b17" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 46805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:43:01.164672 7366 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:43:01.164969 7366 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:43:01.167980 7366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d24ce6efb5f0486199001fd7a19506ad: Sending DeleteTablet for 3 replicas of tablet 982ecb92cffb4c1ea916b2517b18c4c4
I20260808 14:43:01.169052 7637 tablet_service.cc:1514] Processing DeleteTablet for tablet 982ecb92cffb4c1ea916b2517b18c4c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:43:01 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:56650
I20260808 14:43:01.169308 7501 tablet_service.cc:1514] Processing DeleteTablet for tablet 982ecb92cffb4c1ea916b2517b18c4c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:43:01 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:58242
I20260808 14:43:01.169301 7777 tablet_service.cc:1514] Processing DeleteTablet for tablet 982ecb92cffb4c1ea916b2517b18c4c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:43:01 UTC) from {username='kudu', principal='kudu/127.19.144.126@KRBTEST.COM'} at 127.0.0.1:51458
I20260808 14:43:01.169723 7886 tablet_replica.cc:330] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17: stopping tablet replica
I20260808 14:43:01.169849 7887 tablet_replica.cc:330] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d: stopping tablet replica
I20260808 14:43:01.169945 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 7437
I20260808 14:43:01.170130 7886 raft_consensus.cc:2229] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:43:01.170128 7887 raft_consensus.cc:2229] T 982ecb92cffb4c1ea916b2517b18c4c4 P ca031c871ee44f7d8708b131866c267d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:43:01.170486 7886 raft_consensus.cc:2258] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:43:01.172127 7886 ts_tablet_manager.cc:1905] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:43:01.175366 7886 ts_tablet_manager.cc:1918] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:43:01.175519 7886 log.cc:1196] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/982ecb92cffb4c1ea916b2517b18c4c4
I20260808 14:43:01.175909 7886 ts_tablet_manager.cc:1939] T 982ecb92cffb4c1ea916b2517b18c4c4 P d798462d34a7443e9382fc5c43ff8b17: Deleting consensus metadata
I20260808 14:43:01.177675 7356 catalog_manager.cc:4841] TS d798462d34a7443e9382fc5c43ff8b17 (127.19.144.66:46805): tablet 982ecb92cffb4c1ea916b2517b18c4c4 (table test-table [id=c7752fcceefb4b32bc1ff65760f61914]) successfully deleted
I20260808 14:43:01.178157 7888 tablet_replica.cc:330] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2: stopping tablet replica
I20260808 14:43:01.178592 7888 raft_consensus.cc:2229] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:43:01.179224 7888 raft_consensus.cc:2258] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:43:01.181419 7888 ts_tablet_manager.cc:1905] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 14:43:01.182394 7357 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.144.65:40423 (error 108)
W20260808 14:43:01.183794 7857 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:01.183378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:40423 (local address 127.0.0.1:58254)
0808 14:43:01.183536 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:01.183557 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:01.183650 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:40423: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 14:43:01.183974 7357 catalog_manager.cc:4584] TS ca031c871ee44f7d8708b131866c267d (127.19.144.65:40423): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 982ecb92cffb4c1ea916b2517b18c4c4: Network error: Client connection negotiation failed: client connection to 127.19.144.65:40423: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260808 14:43:01.184360 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 7572
I20260808 14:43:01.184386 7888 ts_tablet_manager.cc:1918] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:43:01.184486 7888 log.cc:1196] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/982ecb92cffb4c1ea916b2517b18c4c4
I20260808 14:43:01.184787 7888 ts_tablet_manager.cc:1939] T 982ecb92cffb4c1ea916b2517b18c4c4 P ec6cb6a1afde494ba8642f4db74aa0b2: Deleting consensus metadata
I20260808 14:43:01.187122 7357 catalog_manager.cc:4841] TS ec6cb6a1afde494ba8642f4db74aa0b2 (127.19.144.67:45935): tablet 982ecb92cffb4c1ea916b2517b18c4c4 (table test-table [id=c7752fcceefb4b32bc1ff65760f61914]) successfully deleted
I20260808 14:43:01.193022 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 7710
I20260808 14:43:01.200901 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 7340
2026-08-08T14:43:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2568 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T14:43:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:01Z Disabled control of system clock
I20260808 14:43:01.234872 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:40469
--webserver_interface=127.19.144.126
--webserver_port=0
--builtin_ntp_servers=127.19.144.84:40801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:40469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 14:43:01.352313 7894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:01.352607 7894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:01.352643 7894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:01.356591 7894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:01.356654 7894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:01.356669 7894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:01.356686 7894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:01.356701 7894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:01.361552 7894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:40801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:40469
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:40469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:01.363163 7894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:01.364645 7894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:01.373827 7900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:01.373925 7899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:01.373865 7902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:01.374177 7894 server_base.cc:1034] running on GCE node
I20260808 14:43:01.374828 7894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:01.375960 7894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:01.377125 7894 hybrid_clock.cc:648] HybridClock initialized: now 1786200181377107 us; error 32 us; skew 500 ppm
I20260808 14:43:01.383565 7894 webserver.cc:460] Webserver started at http://127.19.144.126:34005/ using document root <none> and password file <none>
I20260808 14:43:01.384181 7894 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:01.384246 7894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:01.384425 7894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:01.386687 7894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "f10b0b2ccca441829426481a27b74f13"
format_stamp: "Formatted at 2026-08-08 14:43:01 on dist-test-slave-p3t5"
server_key: "e50b1497dfe5fd874a29c1594df4b751"
server_key_iv: "d8e12edece52799d6d41ffffebb5dda3"
server_key_version: "encryptionkey@0"
I20260808 14:43:01.387271 7894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "f10b0b2ccca441829426481a27b74f13"
format_stamp: "Formatted at 2026-08-08 14:43:01 on dist-test-slave-p3t5"
server_key: "e50b1497dfe5fd874a29c1594df4b751"
server_key_iv: "d8e12edece52799d6d41ffffebb5dda3"
server_key_version: "encryptionkey@0"
I20260808 14:43:01.391144 7894 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:43:01.393776 7908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:01.395443 7894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:43:01.395614 7894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "f10b0b2ccca441829426481a27b74f13"
format_stamp: "Formatted at 2026-08-08 14:43:01 on dist-test-slave-p3t5"
server_key: "e50b1497dfe5fd874a29c1594df4b751"
server_key_iv: "d8e12edece52799d6d41ffffebb5dda3"
server_key_version: "encryptionkey@0"
I20260808 14:43:01.395747 7894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:01.409543 7894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:01.410352 7894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:01.410531 7894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:01.418200 7894 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:40469
I20260808 14:43:01.418231 7962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:40469 every 8 connection(s)
I20260808 14:43:01.419230 7894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:01.422169 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 7894
I20260808 14:43:01.422304 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:01.422654 20033 external_mini_cluster.cc:1428] Setting key cf213ebdf5cfd7ad6003eb7367de9d7b
I20260808 14:43:01.422785 7963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:01.430469 7963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13: Bootstrap starting.
I20260808 14:43:01.433144 7963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:01.434039 7963 log.cc:824] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:01.434090 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:01.424229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50900 (local address 127.19.144.126:40469)
0808 14:43:01.424992 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:01.425009 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:01.425048 (+ 39us) server_negotiation.cc:407] Connection header received
0808 14:43:01.425761 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:01.425781 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:01.426203 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:01.426551 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:01.427030 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.428608 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.430078 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.431029 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.431198 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:01.432177 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:01.432182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:01.432196 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:01.432605 (+ 409us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:01.432663 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:01.432685 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:01.432796 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:01.432921 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 14:43:01.433210 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":163,"threads_started":1}
I20260808 14:43:01.436218 7963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13: No bootstrap required, opened a new log
I20260808 14:43:01.440120 7963 raft_consensus.cc:348] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f10b0b2ccca441829426481a27b74f13" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 40469 } }
I20260808 14:43:01.440367 7963 raft_consensus.cc:374] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:01.440405 7963 raft_consensus.cc:729] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f10b0b2ccca441829426481a27b74f13, State: Initialized, Role: FOLLOWER
I20260808 14:43:01.440894 7963 consensus_queue.cc:260] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f10b0b2ccca441829426481a27b74f13" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 40469 } }
I20260808 14:43:01.441085 7963 raft_consensus.cc:388] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:01.441176 7963 raft_consensus.cc:482] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:01.441295 7963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:01.442377 7963 raft_consensus.cc:504] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f10b0b2ccca441829426481a27b74f13" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 40469 } }
I20260808 14:43:01.442761 7963 leader_election.cc:302] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f10b0b2ccca441829426481a27b74f13; no voters:
I20260808 14:43:01.443133 7963 leader_election.cc:288] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:01.443251 7969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:01.443506 7969 raft_consensus.cc:686] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [term 1 LEADER]: Becoming Leader. State: Replica: f10b0b2ccca441829426481a27b74f13, State: Running, Role: LEADER
I20260808 14:43:01.443852 7969 consensus_queue.cc:237] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f10b0b2ccca441829426481a27b74f13" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 40469 } }
I20260808 14:43:01.444478 7963 sys_catalog.cc:564] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:01.445819 7970 sys_catalog.cc:455] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f10b0b2ccca441829426481a27b74f13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f10b0b2ccca441829426481a27b74f13" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 40469 } } }
I20260808 14:43:01.445960 7970 sys_catalog.cc:458] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:01.446303 7971 sys_catalog.cc:455] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f10b0b2ccca441829426481a27b74f13. Latest consensus state: current_term: 1 leader_uuid: "f10b0b2ccca441829426481a27b74f13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f10b0b2ccca441829426481a27b74f13" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 40469 } } }
I20260808 14:43:01.446393 7979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:01.446408 7971 sys_catalog.cc:458] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:01.449780 7979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:01.455761 7979 catalog_manager.cc:1347] Generated new cluster ID: cd4744b1e8bb4db7ae5b860dbbb2ebe7
I20260808 14:43:01.455854 7979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:01.471545 7979 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:01.472584 7979 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:01.484800 7979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13: Generated new TSK 0
I20260808 14:43:01.485669 7979 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:01.495103 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.65:0
--local_ip_for_outbound_sockets=127.19.144.65
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:40469
--builtin_ntp_servers=127.19.144.84:40801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 14:43:01.614022 7991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:01.614347 7991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:01.614423 7991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:01.618543 7991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:01.618629 7991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:01.618722 7991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.65
I20260808 14:43:01.623559 7991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:40801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.144.65
--webserver_port=0
--tserver_master_addrs=127.19.144.126:40469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.65
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:01.624756 7991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:01.626278 7991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:01.636379 8000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:01.636379 7997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:01.636529 7991 server_base.cc:1034] running on GCE node
W20260808 14:43:01.636730 7998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:01.637169 7991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:01.637897 7991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:01.639081 7991 hybrid_clock.cc:648] HybridClock initialized: now 1786200181639045 us; error 49 us; skew 500 ppm
I20260808 14:43:01.641085 7991 webserver.cc:460] Webserver started at http://127.19.144.65:41533/ using document root <none> and password file <none>
I20260808 14:43:01.641760 7991 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:01.641856 7991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:01.642108 7991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:01.644418 7991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/data/instance:
uuid: "d48a112a2d2942bf848176218e48cd02"
format_stamp: "Formatted at 2026-08-08 14:43:01 on dist-test-slave-p3t5"
server_key: "892a8e5624fcb35bf67c4b0ed78c50f3"
server_key_iv: "3441a19896b07c4a8d6f6ae70fe0cf01"
server_key_version: "encryptionkey@0"
I20260808 14:43:01.645058 7991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance:
uuid: "d48a112a2d2942bf848176218e48cd02"
format_stamp: "Formatted at 2026-08-08 14:43:01 on dist-test-slave-p3t5"
server_key: "892a8e5624fcb35bf67c4b0ed78c50f3"
server_key_iv: "3441a19896b07c4a8d6f6ae70fe0cf01"
server_key_version: "encryptionkey@0"
I20260808 14:43:01.649149 7991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260808 14:43:01.651753 8006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:01.653108 7991 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:43:01.653249 7991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/wal
uuid: "d48a112a2d2942bf848176218e48cd02"
format_stamp: "Formatted at 2026-08-08 14:43:01 on dist-test-slave-p3t5"
server_key: "892a8e5624fcb35bf67c4b0ed78c50f3"
server_key_iv: "3441a19896b07c4a8d6f6ae70fe0cf01"
server_key_version: "encryptionkey@0"
I20260808 14:43:01.653373 7991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:01.669795 7991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:01.670562 7991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:01.670773 7991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:01.671401 7991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:43:01.672425 7991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:43:01.672494 7991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:01.672565 7991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:43:01.672613 7991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:01.681643 7991 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.65:39741
I20260808 14:43:01.681766 8122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.65:39741 every 8 connection(s)
I20260808 14:43:01.682739 7991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/data/info.pb
I20260808 14:43:01.683235 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 7991
I20260808 14:43:01.683360 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-0/wal/instance
I20260808 14:43:01.683732 20033 external_mini_cluster.cc:1428] Setting key a300a47c0ed69971dc566124fda67ad9
I20260808 14:43:01.686180 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.66:0
--local_ip_for_outbound_sockets=127.19.144.66
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:40469
--builtin_ntp_servers=127.19.144.84:40801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 14:43:01.694320 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:01.684928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.65:35661 (local address 127.19.144.126:40469)
0808 14:43:01.685078 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:01.685083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:01.686462 (+ 1379us) server_negotiation.cc:407] Connection header received
0808 14:43:01.688112 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:01.688116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:01.688176 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:01.688270 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:01.689676 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.690554 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.691794 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.692239 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.692375 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:01.693671 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:01.693678 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:01.693682 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:01.693704 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:01.693733 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:01.693747 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:01.693839 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:01.694001 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 14:43:01.694096 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:43:01.695240 8126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:01.685480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:40469 (local address 127.19.144.65:35661)
0808 14:43:01.686277 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:01.686343 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:01.687763 (+ 1420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:01.688498 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:01.688510 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:01.688639 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:01.689448 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:01.689476 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.690698 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.690701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:01.691660 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:01.691670 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.692422 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.692425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:01.692595 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:01.693323 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:43:01.693325 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:01.693374 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:43:01.693461 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:01.693837 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:01.693846 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:01.693851 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 14:43:01.693971 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 14:43:01.694251 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":180,"threads_started":1}
I20260808 14:43:01.696630 8123 heartbeater.cc:347] Connected to a master server at 127.19.144.126:40469
I20260808 14:43:01.696893 8123 heartbeater.cc:464] Registering TS with master...
I20260808 14:43:01.697489 8123 heartbeater.cc:511] Master 127.19.144.126:40469 requested a full tablet report, sending...
I20260808 14:43:01.699182 7927 ts_manager.cc:194] Registered new tserver with Master: d48a112a2d2942bf848176218e48cd02 (127.19.144.65:39741)
I20260808 14:43:01.700556 7927 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.144.65:35661
W20260808 14:43:01.807832 8127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:01.808171 8127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:01.808243 8127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:01.812369 8127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:01.812472 8127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:01.812574 8127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.66
I20260808 14:43:01.817582 8127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:40801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.144.66
--webserver_port=0
--tserver_master_addrs=127.19.144.126:40469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.66
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:01.818787 8127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:01.820324 8127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:01.830778 8132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:01.830785 8133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:01.830785 8135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:01.831194 8127 server_base.cc:1034] running on GCE node
I20260808 14:43:01.831640 8127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:01.832405 8127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:01.833590 8127 hybrid_clock.cc:648] HybridClock initialized: now 1786200181833572 us; error 33 us; skew 500 ppm
I20260808 14:43:01.835683 8127 webserver.cc:460] Webserver started at http://127.19.144.66:35591/ using document root <none> and password file <none>
I20260808 14:43:01.836295 8127 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:01.836382 8127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:01.836627 8127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:01.838841 8127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/data/instance:
uuid: "fa3adac0a9284497b389f54b9fc9ef88"
format_stamp: "Formatted at 2026-08-08 14:43:01 on dist-test-slave-p3t5"
server_key: "3f16955577b6e47fdc2529bd87da17e3"
server_key_iv: "4d4a707b75fe88df4834973326d3d5d2"
server_key_version: "encryptionkey@0"
I20260808 14:43:01.839457 8127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance:
uuid: "fa3adac0a9284497b389f54b9fc9ef88"
format_stamp: "Formatted at 2026-08-08 14:43:01 on dist-test-slave-p3t5"
server_key: "3f16955577b6e47fdc2529bd87da17e3"
server_key_iv: "4d4a707b75fe88df4834973326d3d5d2"
server_key_version: "encryptionkey@0"
I20260808 14:43:01.843389 8127 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 14:43:01.846148 8143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:01.847590 8127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:43:01.847738 8127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/wal
uuid: "fa3adac0a9284497b389f54b9fc9ef88"
format_stamp: "Formatted at 2026-08-08 14:43:01 on dist-test-slave-p3t5"
server_key: "3f16955577b6e47fdc2529bd87da17e3"
server_key_iv: "4d4a707b75fe88df4834973326d3d5d2"
server_key_version: "encryptionkey@0"
I20260808 14:43:01.847885 8127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:01.865764 8127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:01.866524 8127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:01.866755 8127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:01.867406 8127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:43:01.868427 8127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:43:01.868476 8127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:01.868546 8127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:43:01.868579 8127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:01.877306 8127 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.66:33553
I20260808 14:43:01.877331 8260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.66:33553 every 8 connection(s)
I20260808 14:43:01.878348 8127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/data/info.pb
I20260808 14:43:01.885499 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 8127
I20260808 14:43:01.885686 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/wal/instance
I20260808 14:43:01.886039 20033 external_mini_cluster.cc:1428] Setting key 153cbf7f5d9cce55f60f0397adf03dc9
I20260808 14:43:01.888147 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.144.67:0
--local_ip_for_outbound_sockets=127.19.144.67
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:40469
--builtin_ntp_servers=127.19.144.84:40801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 14:43:01.888863 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:01.880331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:48741 (local address 127.19.144.126:40469)
0808 14:43:01.880471 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:01.880474 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:01.881596 (+ 1122us) server_negotiation.cc:407] Connection header received
0808 14:43:01.883277 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:01.883280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:01.883345 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:01.883451 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:01.884394 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.884932 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.886654 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.886892 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.887014 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:01.888181 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:01.888185 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:01.888187 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:01.888206 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:01.888233 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:01.888242 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:01.888324 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:01.888556 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 14:43:01.888638 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:43:01.889757 8263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:01.880749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:40469 (local address 127.19.144.66:48741)
0808 14:43:01.881441 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:01.881484 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:01.883042 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:01.883640 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:01.883647 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:01.883744 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:01.884248 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:01.884261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.885062 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.885070 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:01.886517 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:01.886528 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:01.887061 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:01.887066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:01.887243 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:01.887900 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:43:01.887902 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:01.887946 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:43:01.888016 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:01.888369 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:01.888380 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:01.888387 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 14:43:01.888512 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 14:43:01.888764 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":118,"threads_started":1}
I20260808 14:43:01.891364 8261 heartbeater.cc:347] Connected to a master server at 127.19.144.126:40469
I20260808 14:43:01.891621 8261 heartbeater.cc:464] Registering TS with master...
I20260808 14:43:01.892167 8261 heartbeater.cc:511] Master 127.19.144.126:40469 requested a full tablet report, sending...
I20260808 14:43:01.893306 7927 ts_manager.cc:194] Registered new tserver with Master: fa3adac0a9284497b389f54b9fc9ef88 (127.19.144.66:33553)
I20260808 14:43:01.893959 7927 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.144.66:48741
W20260808 14:43:02.010473 8264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:02.010859 8264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:02.010942 8264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:02.015208 8264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:02.015302 8264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:02.015396 8264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.144.67
I20260808 14:43:02.021044 8264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:40801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.144.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.144.67
--webserver_port=0
--tserver_master_addrs=127.19.144.126:40469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.144.67
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:02.022277 8264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:02.023777 8264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:02.033944 8270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:02.033951 8269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:02.033948 8272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:02.034300 8264 server_base.cc:1034] running on GCE node
I20260808 14:43:02.034718 8264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:02.035427 8264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:02.036623 8264 hybrid_clock.cc:648] HybridClock initialized: now 1786200182036602 us; error 30 us; skew 500 ppm
I20260808 14:43:02.038652 8264 webserver.cc:460] Webserver started at http://127.19.144.67:40533/ using document root <none> and password file <none>
I20260808 14:43:02.039268 8264 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:02.039335 8264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:02.039577 8264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:02.041801 8264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/data/instance:
uuid: "29a12308d91f4b919cb5778febcab7be"
format_stamp: "Formatted at 2026-08-08 14:43:02 on dist-test-slave-p3t5"
server_key: "72a7b5004904d864db94fffb53b3deec"
server_key_iv: "0a726ba22d50634ce230440e81f02ea0"
server_key_version: "encryptionkey@0"
I20260808 14:43:02.042402 8264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance:
uuid: "29a12308d91f4b919cb5778febcab7be"
format_stamp: "Formatted at 2026-08-08 14:43:02 on dist-test-slave-p3t5"
server_key: "72a7b5004904d864db94fffb53b3deec"
server_key_iv: "0a726ba22d50634ce230440e81f02ea0"
server_key_version: "encryptionkey@0"
I20260808 14:43:02.046469 8264 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:43:02.048983 8278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:02.050320 8264 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:43:02.050465 8264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/wal
uuid: "29a12308d91f4b919cb5778febcab7be"
format_stamp: "Formatted at 2026-08-08 14:43:02 on dist-test-slave-p3t5"
server_key: "72a7b5004904d864db94fffb53b3deec"
server_key_iv: "0a726ba22d50634ce230440e81f02ea0"
server_key_version: "encryptionkey@0"
I20260808 14:43:02.050590 8264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:02.067375 8264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:02.068085 8264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:02.068284 8264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:02.068898 8264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 14:43:02.069937 8264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 14:43:02.070006 8264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:02.070080 8264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 14:43:02.070127 8264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:02.079212 8264 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.67:39355
I20260808 14:43:02.079243 8397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.67:39355 every 8 connection(s)
I20260808 14:43:02.080226 8264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/data/info.pb
I20260808 14:43:02.086510 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 8264
I20260808 14:43:02.086635 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/wal/instance
I20260808 14:43:02.086941 20033 external_mini_cluster.cc:1428] Setting key 588d9f2a632ef24ef1bed5d17999f4c6
I20260808 14:43:02.090277 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:02.082133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.67:41039 (local address 127.19.144.126:40469)
0808 14:43:02.082282 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:02.082286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:02.083462 (+ 1176us) server_negotiation.cc:407] Connection header received
0808 14:43:02.084957 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:02.084961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:02.085025 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:02.085128 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:02.086126 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.086958 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.088280 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.088571 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.088679 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:02.089853 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:02.089856 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:02.089859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:02.089873 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:02.089895 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:02.089903 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:02.089965 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:02.090103 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 14:43:02.090158 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:43:02.091218 8400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:02.082539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:40469 (local address 127.19.144.67:41039)
0808 14:43:02.083273 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:02.083317 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:02.084700 (+ 1383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:02.085314 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:02.085321 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:02.085416 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:02.085978 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:02.085992 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.087114 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.087117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:02.088156 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:02.088167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.088724 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.088727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:02.088891 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:02.089591 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 14:43:02.089593 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:02.089647 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 14:43:02.089718 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:02.089980 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:02.089987 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:02.089991 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 14:43:02.090089 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 14:43:02.090349 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":120,"threads_started":1}
I20260808 14:43:02.092430 8398 heartbeater.cc:347] Connected to a master server at 127.19.144.126:40469
I20260808 14:43:02.092687 8398 heartbeater.cc:464] Registering TS with master...
I20260808 14:43:02.093248 8398 heartbeater.cc:511] Master 127.19.144.126:40469 requested a full tablet report, sending...
I20260808 14:43:02.094357 7927 ts_manager.cc:194] Registered new tserver with Master: 29a12308d91f4b919cb5778febcab7be (127.19.144.67:39355)
I20260808 14:43:02.094930 7927 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.144.67:41039
I20260808 14:43:02.101033 20033 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 14:43:02.106782 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:02.103142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50912 (local address 127.19.144.126:40469)
0808 14:43:02.103271 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:02.103275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:02.103451 (+ 176us) server_negotiation.cc:407] Connection header received
0808 14:43:02.103562 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:02.103565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:02.103609 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:02.103681 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:02.103932 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.104448 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.105251 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.105467 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.105546 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:02.106396 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:02.106399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:02.106401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:02.106413 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:02.106430 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:02.106438 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:02.106495 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:02.106605 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 14:43:02.106659 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 14:43:02.109858 7927 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 14:43:02.112812 7927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 14:43:02.128134 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:02.121975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.66:33553 (local address 127.0.0.1:39404)
0808 14:43:02.122394 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:02.122408 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:02.122491 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:02.123312 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:02.123318 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:02.123343 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:43:02.123466 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:02.123472 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.124737 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.124741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:02.126506 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:02.126518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.127421 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.127426 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:02.127563 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:02.127635 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 14:43:02.127766 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 14:43:02.127926 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":173,"thread_start_us":114,"threads_started":1}
I20260808 14:43:02.128316 8414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:02.122849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42144 (local address 127.19.144.67:39355)
0808 14:43:02.123392 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:02.123404 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:02.123427 (+ 23us) server_negotiation.cc:407] Connection header received
0808 14:43:02.123497 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:02.123502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:02.123695 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:02.123843 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:43:02.124073 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.125068 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.126429 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.127228 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.127337 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:02.127551 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:02.128060 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 14:43:02.128164 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":153,"threads_started":1}
I20260808 14:43:02.128592 8409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:02.121510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:39741 (local address 127.0.0.1:47442)
0808 14:43:02.121926 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:02.121983 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:02.122125 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:02.123034 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:02.123040 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:02.123068 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:43:02.123241 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:02.123251 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.124863 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.124866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:02.126506 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:02.126518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.127576 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.127582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:02.127764 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:02.127875 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 14:43:02.127943 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 14:43:02.128057 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":320,"thread_start_us":127,"threads_started":1}
I20260808 14:43:02.128697 8410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:02.121728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47442 (local address 127.19.144.65:39741)
0808 14:43:02.122491 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:02.122504 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:02.122531 (+ 27us) server_negotiation.cc:407] Connection header received
0808 14:43:02.122613 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:02.122621 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:02.122846 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:02.123417 (+ 571us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:43:02.123574 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.124744 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.126641 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.127436 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.128172 (+ 736us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:02.128367 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:02.128437 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 14:43:02.128535 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":162,"threads_started":1}
I20260808 14:43:02.129040 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:02.122336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:39355 (local address 127.0.0.1:42144)
0808 14:43:02.123290 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:02.123303 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:02.123382 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:02.123843 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:02.123846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:02.123865 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:43:02.123955 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:02.123961 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.125180 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.125183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:02.126308 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:02.126317 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.127602 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.127605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:02.127767 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:02.127876 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 14:43:02.127954 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 14:43:02.128887 (+ 933us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"mutex_wait_us":930,"spinlock_wait_cycles":768,"thread_start_us":82,"threads_started":1}
I20260808 14:43:02.129340 8413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:02.122443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39404 (local address 127.19.144.66:33553)
0808 14:43:02.122730 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:02.122744 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:02.122770 (+ 26us) server_negotiation.cc:407] Connection header received
0808 14:43:02.122847 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:02.122852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:02.123124 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:02.123599 (+ 475us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:43:02.123708 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.124619 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.126668 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.127261 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.128804 (+ 1543us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:02.128992 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:02.129046 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 14:43:02.129145 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":103,"threads_started":1}
I20260808 14:43:02.131776 8329 tablet_service.cc:1467] Processing CreateTablet for tablet 28ce95be422c43f788aca7aed7806d72 (DEFAULT_TABLE table=test-table [id=af0d69d8a2c04917bd9582aab47c5b67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:43:02.131871 8191 tablet_service.cc:1467] Processing CreateTablet for tablet 28ce95be422c43f788aca7aed7806d72 (DEFAULT_TABLE table=test-table [id=af0d69d8a2c04917bd9582aab47c5b67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:43:02.131873 8055 tablet_service.cc:1467] Processing CreateTablet for tablet 28ce95be422c43f788aca7aed7806d72 (DEFAULT_TABLE table=test-table [id=af0d69d8a2c04917bd9582aab47c5b67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 14:43:02.132874 8329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28ce95be422c43f788aca7aed7806d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:02.132923 8191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28ce95be422c43f788aca7aed7806d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:02.132923 8055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28ce95be422c43f788aca7aed7806d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:02.139571 8415 tablet_bootstrap.cc:492] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be: Bootstrap starting.
I20260808 14:43:02.141167 8417 tablet_bootstrap.cc:492] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88: Bootstrap starting.
I20260808 14:43:02.142417 8415 tablet_bootstrap.cc:654] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:02.143168 8415 log.cc:824] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:02.143199 8417 tablet_bootstrap.cc:654] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:02.143888 8417 log.cc:824] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:02.144583 8416 tablet_bootstrap.cc:492] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02: Bootstrap starting.
I20260808 14:43:02.145388 8415 tablet_bootstrap.cc:492] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be: No bootstrap required, opened a new log
I20260808 14:43:02.145599 8415 ts_tablet_manager.cc:1397] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 14:43:02.146701 8416 tablet_bootstrap.cc:654] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:02.147650 8416 log.cc:824] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:02.147712 8417 tablet_bootstrap.cc:492] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88: No bootstrap required, opened a new log
I20260808 14:43:02.147912 8417 ts_tablet_manager.cc:1397] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260808 14:43:02.149700 8415 raft_consensus.cc:348] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48a112a2d2942bf848176218e48cd02" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 39741 } } peers { permanent_uuid: "29a12308d91f4b919cb5778febcab7be" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39355 } } peers { permanent_uuid: "fa3adac0a9284497b389f54b9fc9ef88" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33553 } }
I20260808 14:43:02.149914 8416 tablet_bootstrap.cc:492] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02: No bootstrap required, opened a new log
I20260808 14:43:02.149957 8415 raft_consensus.cc:374] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:02.150030 8415 raft_consensus.cc:729] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a12308d91f4b919cb5778febcab7be, State: Initialized, Role: FOLLOWER
I20260808 14:43:02.150103 8416 ts_tablet_manager.cc:1397] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 14:43:02.150528 8415 consensus_queue.cc:260] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48a112a2d2942bf848176218e48cd02" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 39741 } } peers { permanent_uuid: "29a12308d91f4b919cb5778febcab7be" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39355 } } peers { permanent_uuid: "fa3adac0a9284497b389f54b9fc9ef88" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33553 } }
I20260808 14:43:02.151221 8398 heartbeater.cc:503] Master 127.19.144.126:40469 was elected leader, sending a full tablet report...
I20260808 14:43:02.151660 8415 ts_tablet_manager.cc:1428] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260808 14:43:02.152220 8417 raft_consensus.cc:348] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48a112a2d2942bf848176218e48cd02" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 39741 } } peers { permanent_uuid: "29a12308d91f4b919cb5778febcab7be" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39355 } } peers { permanent_uuid: "fa3adac0a9284497b389f54b9fc9ef88" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33553 } }
I20260808 14:43:02.152509 8417 raft_consensus.cc:374] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:02.152622 8417 raft_consensus.cc:729] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3adac0a9284497b389f54b9fc9ef88, State: Initialized, Role: FOLLOWER
I20260808 14:43:02.153118 8417 consensus_queue.cc:260] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48a112a2d2942bf848176218e48cd02" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 39741 } } peers { permanent_uuid: "29a12308d91f4b919cb5778febcab7be" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39355 } } peers { permanent_uuid: "fa3adac0a9284497b389f54b9fc9ef88" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33553 } }
I20260808 14:43:02.154201 8417 ts_tablet_manager.cc:1428] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260808 14:43:02.154574 8261 heartbeater.cc:503] Master 127.19.144.126:40469 was elected leader, sending a full tablet report...
I20260808 14:43:02.154453 8416 raft_consensus.cc:348] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48a112a2d2942bf848176218e48cd02" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 39741 } } peers { permanent_uuid: "29a12308d91f4b919cb5778febcab7be" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39355 } } peers { permanent_uuid: "fa3adac0a9284497b389f54b9fc9ef88" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33553 } }
I20260808 14:43:02.154731 8416 raft_consensus.cc:374] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:02.154791 8416 raft_consensus.cc:729] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d48a112a2d2942bf848176218e48cd02, State: Initialized, Role: FOLLOWER
I20260808 14:43:02.155251 8416 consensus_queue.cc:260] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48a112a2d2942bf848176218e48cd02" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 39741 } } peers { permanent_uuid: "29a12308d91f4b919cb5778febcab7be" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39355 } } peers { permanent_uuid: "fa3adac0a9284497b389f54b9fc9ef88" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33553 } }
I20260808 14:43:02.156315 8416 ts_tablet_manager.cc:1428] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 14:43:02.156617 8123 heartbeater.cc:503] Master 127.19.144.126:40469 was elected leader, sending a full tablet report...
I20260808 14:43:02.173907 8422 raft_consensus.cc:482] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 14:43:02.174149 8422 raft_consensus.cc:504] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48a112a2d2942bf848176218e48cd02" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 39741 } } peers { permanent_uuid: "29a12308d91f4b919cb5778febcab7be" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39355 } } peers { permanent_uuid: "fa3adac0a9284497b389f54b9fc9ef88" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33553 } }
I20260808 14:43:02.175457 8422 leader_election.cc:288] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d48a112a2d2942bf848176218e48cd02 (127.19.144.65:39741), 29a12308d91f4b919cb5778febcab7be (127.19.144.67:39355)
I20260808 14:43:02.180563 8414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:02.176163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:34363 (local address 127.19.144.67:39355)
0808 14:43:02.176293 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:02.176298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:02.176557 (+ 259us) server_negotiation.cc:407] Connection header received
0808 14:43:02.176657 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:02.176660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:02.176718 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:02.176809 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:43:02.177068 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.177782 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.179180 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.179807 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.179941 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:02.179997 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:02.180346 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 14:43:02.180419 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 14:43:02.180563 8410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:02.175821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.144.66:39795 (local address 127.19.144.65:39741)
0808 14:43:02.175992 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:02.175997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:02.176016 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:43:02.176080 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:02.176083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:02.176145 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:02.176234 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 14:43:02.176675 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.177496 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.179180 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.179807 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.179941 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:02.179997 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:02.180341 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 14:43:02.180419 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 14:43:02.180624 8263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:02.175707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:39741 (local address 127.19.144.66:39795)
0808 14:43:02.175863 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:02.175882 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:02.175993 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:02.176370 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:02.176373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:02.176395 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:43:02.176536 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:02.176543 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.177684 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.177693 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:02.179027 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:02.179040 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.179952 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.179956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:02.180140 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:02.180222 (+ 82us) client_negotiation.cc:769] Sending connection context
0808 14:43:02.180309 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 14:43:02.180445 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":94}
I20260808 14:43:02.180634 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:02.176051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.67:39355 (local address 127.19.144.66:34363)
0808 14:43:02.176416 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:02.176430 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:02.176542 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:02.176831 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:02.176833 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:02.176851 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 14:43:02.176955 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:02.176959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.177887 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.177891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:02.179027 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:02.179040 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.179952 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.179956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:02.180165 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:02.180230 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 14:43:02.180308 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 14:43:02.180445 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":99,"thread_start_us":153,"threads_started":1}
I20260808 14:43:02.181584 8350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28ce95be422c43f788aca7aed7806d72" candidate_uuid: "fa3adac0a9284497b389f54b9fc9ef88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a12308d91f4b919cb5778febcab7be" is_pre_election: true
I20260808 14:43:02.181583 8077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28ce95be422c43f788aca7aed7806d72" candidate_uuid: "fa3adac0a9284497b389f54b9fc9ef88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d48a112a2d2942bf848176218e48cd02" is_pre_election: true
I20260808 14:43:02.181977 8077 raft_consensus.cc:2454] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3adac0a9284497b389f54b9fc9ef88 in term 0.
I20260808 14:43:02.181977 8350 raft_consensus.cc:2454] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3adac0a9284497b389f54b9fc9ef88 in term 0.
I20260808 14:43:02.182507 8147 leader_election.cc:302] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29a12308d91f4b919cb5778febcab7be, fa3adac0a9284497b389f54b9fc9ef88; no voters:
I20260808 14:43:02.182816 8422 raft_consensus.cc:2781] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 14:43:02.182929 8422 raft_consensus.cc:482] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 14:43:02.183018 8422 raft_consensus.cc:3037] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [term 0 FOLLOWER]: Advancing to term 1
W20260808 14:43:02.184005 8124 tablet.cc:2369] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 14:43:02.184188 8422 raft_consensus.cc:504] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48a112a2d2942bf848176218e48cd02" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 39741 } } peers { permanent_uuid: "29a12308d91f4b919cb5778febcab7be" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39355 } } peers { permanent_uuid: "fa3adac0a9284497b389f54b9fc9ef88" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33553 } }
I20260808 14:43:02.184558 8422 leader_election.cc:288] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [CANDIDATE]: Term 1 election: Requested vote from peers d48a112a2d2942bf848176218e48cd02 (127.19.144.65:39741), 29a12308d91f4b919cb5778febcab7be (127.19.144.67:39355)
I20260808 14:43:02.185030 8077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28ce95be422c43f788aca7aed7806d72" candidate_uuid: "fa3adac0a9284497b389f54b9fc9ef88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d48a112a2d2942bf848176218e48cd02"
I20260808 14:43:02.185073 8350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28ce95be422c43f788aca7aed7806d72" candidate_uuid: "fa3adac0a9284497b389f54b9fc9ef88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a12308d91f4b919cb5778febcab7be"
I20260808 14:43:02.185182 8077 raft_consensus.cc:3037] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:02.185189 8350 raft_consensus.cc:3037] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:02.186571 8350 raft_consensus.cc:2454] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3adac0a9284497b389f54b9fc9ef88 in term 1.
I20260808 14:43:02.186571 8077 raft_consensus.cc:2454] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3adac0a9284497b389f54b9fc9ef88 in term 1.
I20260808 14:43:02.186924 8147 leader_election.cc:302] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29a12308d91f4b919cb5778febcab7be, fa3adac0a9284497b389f54b9fc9ef88; no voters:
I20260808 14:43:02.187100 8422 raft_consensus.cc:2781] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:02.187376 8422 raft_consensus.cc:686] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [term 1 LEADER]: Becoming Leader. State: Replica: fa3adac0a9284497b389f54b9fc9ef88, State: Running, Role: LEADER
I20260808 14:43:02.187777 8422 consensus_queue.cc:237] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48a112a2d2942bf848176218e48cd02" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 39741 } } peers { permanent_uuid: "29a12308d91f4b919cb5778febcab7be" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39355 } } peers { permanent_uuid: "fa3adac0a9284497b389f54b9fc9ef88" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33553 } }
I20260808 14:43:02.191427 7926 catalog_manager.cc:5495] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa3adac0a9284497b389f54b9fc9ef88 (127.19.144.66). New cstate: current_term: 1 leader_uuid: "fa3adac0a9284497b389f54b9fc9ef88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48a112a2d2942bf848176218e48cd02" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 39741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29a12308d91f4b919cb5778febcab7be" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3adac0a9284497b389f54b9fc9ef88" member_type: VOTER last_known_addr { host: "127.19.144.66" port: 33553 } health_report { overall_health: HEALTHY } } }
I20260808 14:43:02.212545 8413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:02.208929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39410 (local address 127.19.144.66:33553)
0808 14:43:02.209085 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:02.209091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:02.209110 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:43:02.209201 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:02.209204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:02.209259 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:02.209371 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:43:02.209662 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.210175 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.211024 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:02.211249 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:02.211351 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:02.211679 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:43:02.212131 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:43:02.212268 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:02.212308 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 14:43:02.212365 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 14:43:02.218928 8077 raft_consensus.cc:1261] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02 [term 1 FOLLOWER]: Refusing update from remote peer fa3adac0a9284497b389f54b9fc9ef88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:43:02.218930 8350 raft_consensus.cc:1261] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be [term 1 FOLLOWER]: Refusing update from remote peer fa3adac0a9284497b389f54b9fc9ef88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:43:02.219731 8422 consensus_queue.cc:1035] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a12308d91f4b919cb5778febcab7be" member_type: VOTER last_known_addr { host: "127.19.144.67" port: 39355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:43:02.220041 8425 consensus_queue.cc:1035] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d48a112a2d2942bf848176218e48cd02" member_type: VOTER last_known_addr { host: "127.19.144.65" port: 39741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:43:02.227443 8429 mvcc.cc:205] Tried to move back new op lower bound from 7316275946360578048 to 7316275946244481024. Current Snapshot: MvccSnapshot[applied={T|T < 7316275946360578048}]
I20260808 14:43:02.227666 8431 mvcc.cc:205] Tried to move back new op lower bound from 7316275946360578048 to 7316275946244481024. Current Snapshot: MvccSnapshot[applied={T|T < 7316275946360578048}]
I20260808 14:43:02.229920 8430 mvcc.cc:205] Tried to move back new op lower bound from 7316275946360578048 to 7316275946244481024. Current Snapshot: MvccSnapshot[applied={T|T < 7316275946360578048}]
I20260808 14:43:02.238366 7927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:43:02.238546 7927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 14:43:02.241153 7927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f10b0b2ccca441829426481a27b74f13: Sending DeleteTablet for 3 replicas of tablet 28ce95be422c43f788aca7aed7806d72
I20260808 14:43:02.242121 8055 tablet_service.cc:1514] Processing DeleteTablet for tablet 28ce95be422c43f788aca7aed7806d72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:43:02 UTC) from {username='slave'} at 127.0.0.1:47442
I20260808 14:43:02.242216 8191 tablet_service.cc:1514] Processing DeleteTablet for tablet 28ce95be422c43f788aca7aed7806d72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:43:02 UTC) from {username='slave'} at 127.0.0.1:39404
I20260808 14:43:02.242723 8438 tablet_replica.cc:330] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02: stopping tablet replica
I20260808 14:43:02.242764 8329 tablet_service.cc:1514] Processing DeleteTablet for tablet 28ce95be422c43f788aca7aed7806d72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 14:43:02 UTC) from {username='slave'} at 127.0.0.1:42144
I20260808 14:43:02.243116 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 7991
I20260808 14:43:02.243094 8438 raft_consensus.cc:2229] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:43:02.243474 8438 raft_consensus.cc:2258] T 28ce95be422c43f788aca7aed7806d72 P d48a112a2d2942bf848176218e48cd02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:43:02.243896 8439 tablet_replica.cc:330] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88: stopping tablet replica
I20260808 14:43:02.244242 8439 raft_consensus.cc:2229] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [term 1 LEADER]: Raft consensus shutting down.
I20260808 14:43:02.244765 8439 raft_consensus.cc:2258] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:43:02.246747 8439 ts_tablet_manager.cc:1905] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:43:02.247380 8440 tablet_replica.cc:330] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be: stopping tablet replica
I20260808 14:43:02.247732 8440 raft_consensus.cc:2229] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 14:43:02.248108 8440 raft_consensus.cc:2258] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 14:43:02.250175 8439 ts_tablet_manager.cc:1918] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:43:02.250293 8439 log.cc:1196] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-1/wal/wals/28ce95be422c43f788aca7aed7806d72
I20260808 14:43:02.250659 8439 ts_tablet_manager.cc:1939] T 28ce95be422c43f788aca7aed7806d72 P fa3adac0a9284497b389f54b9fc9ef88: Deleting consensus metadata
I20260808 14:43:02.250614 8440 ts_tablet_manager.cc:1905] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 14:43:02.252362 7911 catalog_manager.cc:4841] TS fa3adac0a9284497b389f54b9fc9ef88 (127.19.144.66:33553): tablet 28ce95be422c43f788aca7aed7806d72 (table test-table [id=af0d69d8a2c04917bd9582aab47c5b67]) successfully deleted
W20260808 14:43:02.252923 7912 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.144.65:39741 (error 108)
I20260808 14:43:02.254055 8440 ts_tablet_manager.cc:1918] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 14:43:02.254101 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:02.253743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.65:39741 (local address 127.0.0.1:47458)
0808 14:43:02.253915 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:02.254004 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.65:39741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 14:43:02.254161 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 8127
I20260808 14:43:02.254205 8440 log.cc:1196] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be: Deleting WAL directory at /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786199922348708-20033-0/minicluster-data/ts-2/wal/wals/28ce95be422c43f788aca7aed7806d72
I20260808 14:43:02.255062 8440 ts_tablet_manager.cc:1939] T 28ce95be422c43f788aca7aed7806d72 P 29a12308d91f4b919cb5778febcab7be: Deleting consensus metadata
W20260808 14:43:02.256511 7912 catalog_manager.cc:4584] TS d48a112a2d2942bf848176218e48cd02 (127.19.144.65:39741): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28ce95be422c43f788aca7aed7806d72: Network error: Client connection negotiation failed: client connection to 127.19.144.65:39741: connect: Connection refused (error 111)
I20260808 14:43:02.256772 7912 catalog_manager.cc:4841] TS 29a12308d91f4b919cb5778febcab7be (127.19.144.67:39355): tablet 28ce95be422c43f788aca7aed7806d72 (table test-table [id=af0d69d8a2c04917bd9582aab47c5b67]) successfully deleted
I20260808 14:43:02.262485 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 8264
I20260808 14:43:02.269898 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 7894
2026-08-08T14:43:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1069 ms)
[----------] 6 tests from EncryptionPolicyTest (11272 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:02 dist-test-slave-p3t5 krb5kdc[8444](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:02 dist-test-slave-p3t5 krb5kdc[8444](info): set up 2 sockets
Aug 08 14:43:02 dist-test-slave-p3t5 krb5kdc[8444](info): commencing operation
krb5kdc: starting...
W20260808 14:43:03.346915 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:03 dist-test-slave-p3t5 krb5kdc[8444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:43:03.486910 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33031
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:03.604246 8457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:03.604606 8457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:03.604684 8457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:03.608913 8457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:03.609042 8457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:03.609078 8457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:03.609122 8457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:03.609166 8457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:03.614830 8457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33031
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:03.616062 8457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:03.617172 8457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:03.626183 8462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:03.626199 8465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:03.626330 8457 server_base.cc:1034] running on GCE node
W20260808 14:43:03.626183 8463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:03.627179 8457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:03.628319 8457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:03.629499 8457 hybrid_clock.cc:648] HybridClock initialized: now 1786200183629468 us; error 47 us; skew 500 ppm
Aug 08 14:43:03 dist-test-slave-p3t5 krb5kdc[8444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200183, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:03.632477 8457 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:43:03.633426 8457 webserver.cc:460] Webserver started at http://127.0.0.1:39041/ using document root <none> and password file <none>
I20260808 14:43:03.634096 8457 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:03.634171 8457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:03.634356 8457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:03.636516 8457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "c3604a9e04004a9a9ee18cab908e560d"
format_stamp: "Formatted at 2026-08-08 14:43:03 on dist-test-slave-p3t5"
server_key: "f0c102c169ae72d9ba1605bd1c5a05b0"
server_key_iv: "7eca27221e2e0322fcd841333852c3ac"
server_key_version: "encryptionkey@0"
I20260808 14:43:03.637082 8457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "c3604a9e04004a9a9ee18cab908e560d"
format_stamp: "Formatted at 2026-08-08 14:43:03 on dist-test-slave-p3t5"
server_key: "f0c102c169ae72d9ba1605bd1c5a05b0"
server_key_iv: "7eca27221e2e0322fcd841333852c3ac"
server_key_version: "encryptionkey@0"
I20260808 14:43:03.641043 8457 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:43:03.643584 8472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:03.644929 8457 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:43:03.645098 8457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "c3604a9e04004a9a9ee18cab908e560d"
format_stamp: "Formatted at 2026-08-08 14:43:03 on dist-test-slave-p3t5"
server_key: "f0c102c169ae72d9ba1605bd1c5a05b0"
server_key_iv: "7eca27221e2e0322fcd841333852c3ac"
server_key_version: "encryptionkey@0"
I20260808 14:43:03.645213 8457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:03.662384 8457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:03.665834 8457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:03.666067 8457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:03.673933 8457 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33031
I20260808 14:43:03.673966 8525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33031 every 8 connection(s)
I20260808 14:43:03.675042 8457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:03.678061 8526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:03.684212 8526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d: Bootstrap starting.
I20260808 14:43:03.685034 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 8457
I20260808 14:43:03.685186 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:03.685626 20033 external_mini_cluster.cc:1428] Setting key daeb28eb438458f3903c2f9736702f9a
I20260808 14:43:03.687901 8526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:03.688760 8526 log.cc:824] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:03.691303 8526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d: No bootstrap required, opened a new log
Aug 08 14:43:03 dist-test-slave-p3t5 krb5kdc[8444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 14:43:03.696339 8526 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3604a9e04004a9a9ee18cab908e560d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33031 } }
I20260808 14:43:03.696707 8526 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:03.696794 8526 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3604a9e04004a9a9ee18cab908e560d, State: Initialized, Role: FOLLOWER
I20260808 14:43:03.697332 8526 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3604a9e04004a9a9ee18cab908e560d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33031 } }
I20260808 14:43:03.697494 8526 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:03.697580 8526 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:03.697708 8526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:03.698733 8526 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3604a9e04004a9a9ee18cab908e560d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33031 } }
I20260808 14:43:03.699110 8526 leader_election.cc:302] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3604a9e04004a9a9ee18cab908e560d; no voters:
I20260808 14:43:03.699461 8526 leader_election.cc:288] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:03.699589 8531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:03.699824 8531 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [term 1 LEADER]: Becoming Leader. State: Replica: c3604a9e04004a9a9ee18cab908e560d, State: Running, Role: LEADER
I20260808 14:43:03.700151 8531 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3604a9e04004a9a9ee18cab908e560d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33031 } }
I20260808 14:43:03.700942 8526 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:03.701484 8533 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3604a9e04004a9a9ee18cab908e560d. Latest consensus state: current_term: 1 leader_uuid: "c3604a9e04004a9a9ee18cab908e560d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3604a9e04004a9a9ee18cab908e560d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33031 } } }
I20260808 14:43:03.701642 8533 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:03.701926 8532 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3604a9e04004a9a9ee18cab908e560d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3604a9e04004a9a9ee18cab908e560d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33031 } } }
I20260808 14:43:03.702023 8532 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:03.702086 8537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:03.702916 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:03.687288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56835 (local address 127.0.0.1:33031)
0808 14:43:03.687792 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:03.687808 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:03.687847 (+ 39us) server_negotiation.cc:407] Connection header received
0808 14:43:03.688615 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:03.688634 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:03.689111 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:03.689536 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:03.690497 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:03.691581 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:03.692751 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:03.693293 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:03.693427 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:03.696689 (+ 3262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:03.696711 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:03.696733 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:03.696771 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:03.698776 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:03.699385 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:03.699392 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:03.699398 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:03.699589 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:03.700008 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:03.700012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:03.700013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:03.700562 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:03.700795 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:03.701456 (+ 661us) server_negotiation.cc:299] Negotiation successful
0808 14:43:03.701751 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":163,"threads_started":1}
I20260808 14:43:03.705775 8537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:03.706809 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:03.712332 8537 catalog_manager.cc:1347] Generated new cluster ID: 7614a95b598b4008b1e32c691f246a77
I20260808 14:43:03.712414 8537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:03.717157 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:03.708935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34679 (local address 127.0.0.1:33031)
0808 14:43:03.709039 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:03.709043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:03.709314 (+ 271us) server_negotiation.cc:407] Connection header received
0808 14:43:03.709427 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:03.709430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:03.709480 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:03.709567 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:03.710408 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:03.711011 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:03.711828 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:03.712063 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:03.712165 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:03.713689 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:03.713693 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:03.713695 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:03.713723 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:03.715260 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:03.715760 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:03.715764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:03.715766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:03.715849 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:03.716182 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:03.716185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:03.716187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:03.716368 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:03.716526 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:03.716864 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 14:43:03.716967 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 14:43:03.721582 8537 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:03.722536 8537 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:03.737103 8537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3604a9e04004a9a9ee18cab908e560d: Generated new TSK 0
I20260808 14:43:03.738010 8537 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:03.746968 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 8457
W20260808 14:43:03.755016 8442 connection.cc:535] client connection to 127.0.0.1:33031 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33031): Connection reset by peer (error 104)
2026-08-08T14:43:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1475 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:03 dist-test-slave-p3t5 krb5kdc[8560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:03 dist-test-slave-p3t5 krb5kdc[8560](info): set up 2 sockets
Aug 08 14:43:03 dist-test-slave-p3t5 krb5kdc[8560](info): commencing operation
krb5kdc: starting...
W20260808 14:43:04.827497 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:04 dist-test-slave-p3t5 krb5kdc[8560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:43:04.963352 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37221
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:05.088438 8574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:05.088766 8574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:05.088966 8574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:05.093674 8574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:05.093762 8574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:05.093783 8574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:05.093801 8574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:05.093863 8574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:05.099639 8574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37221
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:05.100957 8574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:05.102123 8574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:05.111344 8582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:05.111362 8579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:05.111495 8574 server_base.cc:1034] running on GCE node
W20260808 14:43:05.111685 8580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:05.112385 8574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:05.113601 8574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:05.114862 8574 hybrid_clock.cc:648] HybridClock initialized: now 1786200185114846 us; error 31 us; skew 500 ppm
Aug 08 14:43:05 dist-test-slave-p3t5 krb5kdc[8560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200185, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:05.118227 8574 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:43:05.119338 8574 webserver.cc:460] Webserver started at http://127.0.0.1:39925/ using document root <none> and password file <none>
I20260808 14:43:05.119997 8574 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:05.120103 8574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:05.120373 8574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:05.122785 8574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "0d7493b033bf457dbefdbfb6aecafbb6"
format_stamp: "Formatted at 2026-08-08 14:43:05 on dist-test-slave-p3t5"
server_key: "8605d004e380cb9a6bc0f8bb8ffd91c6"
server_key_iv: "ed8c53f01da5ec0ee0d645f91f1fc925"
server_key_version: "encryptionkey@0"
I20260808 14:43:05.123421 8574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "0d7493b033bf457dbefdbfb6aecafbb6"
format_stamp: "Formatted at 2026-08-08 14:43:05 on dist-test-slave-p3t5"
server_key: "8605d004e380cb9a6bc0f8bb8ffd91c6"
server_key_iv: "ed8c53f01da5ec0ee0d645f91f1fc925"
server_key_version: "encryptionkey@0"
I20260808 14:43:05.127657 8574 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:43:05.130553 8589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:05.131923 8574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 14:43:05.132107 8574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "0d7493b033bf457dbefdbfb6aecafbb6"
format_stamp: "Formatted at 2026-08-08 14:43:05 on dist-test-slave-p3t5"
server_key: "8605d004e380cb9a6bc0f8bb8ffd91c6"
server_key_iv: "ed8c53f01da5ec0ee0d645f91f1fc925"
server_key_version: "encryptionkey@0"
I20260808 14:43:05.132246 8574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:05.148874 8574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:05.152895 8574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:05.153133 8574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:05.161026 8574 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37221
I20260808 14:43:05.161082 8642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37221 every 8 connection(s)
I20260808 14:43:05.162258 8574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:05.165406 8643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:05.171672 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 8574
I20260808 14:43:05.171739 8643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6: Bootstrap starting.
I20260808 14:43:05.171864 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:05.172158 20033 external_mini_cluster.cc:1428] Setting key ac2ffa2ec9aae1b041ead291a5d7bbec
I20260808 14:43:05.174998 8643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:05.175995 8643 log.cc:824] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:05.178468 8643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6: No bootstrap required, opened a new log
I20260808 14:43:05.182196 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:05.173658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42095 (local address 127.0.0.1:37221)
0808 14:43:05.174147 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:05.174163 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:05.174203 (+ 40us) server_negotiation.cc:407] Connection header received
0808 14:43:05.174893 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:05.174911 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:05.175346 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:05.175661 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:05.176119 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:05.177224 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:05.178503 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:05.179069 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:05.179198 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:05.180231 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:05.180236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:05.180250 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:05.180666 (+ 416us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:05.180724 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:05.180748 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:05.180885 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:05.181110 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 14:43:05.181273 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":156,"threads_started":1}
I20260808 14:43:05.183136 8643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7493b033bf457dbefdbfb6aecafbb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37221 } }
I20260808 14:43:05.183452 8643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:05.183545 8643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d7493b033bf457dbefdbfb6aecafbb6, State: Initialized, Role: FOLLOWER
I20260808 14:43:05.184077 8643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7493b033bf457dbefdbfb6aecafbb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37221 } }
I20260808 14:43:05.184247 8643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:05.184322 8643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:05.184437 8643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:05.185652 8643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7493b033bf457dbefdbfb6aecafbb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37221 } }
I20260808 14:43:05.185712 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:05.186046 8643 leader_election.cc:302] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d7493b033bf457dbefdbfb6aecafbb6; no voters:
I20260808 14:43:05.186414 8643 leader_election.cc:288] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:05.186554 8648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:05.186808 8648 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [term 1 LEADER]: Becoming Leader. State: Replica: 0d7493b033bf457dbefdbfb6aecafbb6, State: Running, Role: LEADER
I20260808 14:43:05.187268 8648 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7493b033bf457dbefdbfb6aecafbb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37221 } }
I20260808 14:43:05.187885 8643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:05.189292 8649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d7493b033bf457dbefdbfb6aecafbb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7493b033bf457dbefdbfb6aecafbb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37221 } } }
I20260808 14:43:05.189435 8649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:05.189803 8650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d7493b033bf457dbefdbfb6aecafbb6. Latest consensus state: current_term: 1 leader_uuid: "0d7493b033bf457dbefdbfb6aecafbb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d7493b033bf457dbefdbfb6aecafbb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37221 } } }
I20260808 14:43:05.189888 8663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:05.189920 8650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:05.193487 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:05.188208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36949 (local address 127.0.0.1:37221)
0808 14:43:05.188328 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:05.188333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:05.188712 (+ 379us) server_negotiation.cc:407] Connection header received
0808 14:43:05.189007 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:05.189010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:05.189061 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:05.189138 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:05.189514 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:05.190348 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:05.191537 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:05.191849 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:05.191976 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:05.192950 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:05.192954 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:05.192957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:05.192972 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:05.193005 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:05.193019 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:05.193111 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:05.193296 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 14:43:05.193336 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:43:05.193560 8663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:05.199920 8663 catalog_manager.cc:1347] Generated new cluster ID: 51d739765c5e466895e01fee404c1b73
I20260808 14:43:05.200008 8663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:05.213147 8663 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:05.214250 8663 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:05.228205 8663 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d7493b033bf457dbefdbfb6aecafbb6: Generated new TSK 0
I20260808 14:43:05.229059 8663 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:05.238276 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 8574
W20260808 14:43:05.246604 8558 connection.cc:535] client connection to 127.0.0.1:37221 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37221): Connection reset by peer (error 104)
2026-08-08T14:43:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:05 dist-test-slave-p3t5 krb5kdc[8675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:05 dist-test-slave-p3t5 krb5kdc[8675](info): set up 2 sockets
Aug 08 14:43:05 dist-test-slave-p3t5 krb5kdc[8675](info): commencing operation
krb5kdc: starting...
W20260808 14:43:06.303701 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:06 dist-test-slave-p3t5 krb5kdc[8675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:43:06.445506 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45101
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:06.567963 8689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:06.568315 8689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:06.568405 8689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:06.572778 8689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:06.572893 8689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:06.572934 8689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:06.572973 8689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:06.573002 8689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:06.578702 8689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45101
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:06.579932 8689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:06.581058 8689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:06.590118 8694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:06.590107 8695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:06.590107 8697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:06.590408 8689 server_base.cc:1034] running on GCE node
I20260808 14:43:06.591115 8689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:06.592325 8689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:06.593549 8689 hybrid_clock.cc:648] HybridClock initialized: now 1786200186593529 us; error 40 us; skew 500 ppm
Aug 08 14:43:06 dist-test-slave-p3t5 krb5kdc[8675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200186, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:06.601074 8689 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:43:06.602193 8689 webserver.cc:460] Webserver started at http://127.0.0.1:34651/ using document root <none> and password file <none>
I20260808 14:43:06.602824 8689 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:06.602926 8689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:06.603184 8689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:06.605535 8689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "6bdf40211fad43bc80c2e64fdd680a75"
format_stamp: "Formatted at 2026-08-08 14:43:06 on dist-test-slave-p3t5"
server_key: "11c464978532edee2058c6d273685a0e"
server_key_iv: "33a766f122a53487913dc473c74bd332"
server_key_version: "encryptionkey@0"
I20260808 14:43:06.606238 8689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "6bdf40211fad43bc80c2e64fdd680a75"
format_stamp: "Formatted at 2026-08-08 14:43:06 on dist-test-slave-p3t5"
server_key: "11c464978532edee2058c6d273685a0e"
server_key_iv: "33a766f122a53487913dc473c74bd332"
server_key_version: "encryptionkey@0"
I20260808 14:43:06.610544 8689 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:43:06.613302 8704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:06.614646 8689 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 14:43:06.614809 8689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "6bdf40211fad43bc80c2e64fdd680a75"
format_stamp: "Formatted at 2026-08-08 14:43:06 on dist-test-slave-p3t5"
server_key: "11c464978532edee2058c6d273685a0e"
server_key_iv: "33a766f122a53487913dc473c74bd332"
server_key_version: "encryptionkey@0"
I20260808 14:43:06.614950 8689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:06.619903 8689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:06.623219 8689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:06.623445 8689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:06.631254 8689 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45101
I20260808 14:43:06.631284 8757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45101 every 8 connection(s)
I20260808 14:43:06.632282 8689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:06.634353 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 8689
I20260808 14:43:06.634532 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:06.634853 20033 external_mini_cluster.cc:1428] Setting key 3bee4ebdaf18c7c40a72ecf859427024
I20260808 14:43:06.635635 8758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:06.640414 8761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:06.636338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45659 (local address 127.0.0.1:45101)
0808 14:43:06.636818 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:06.636835 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:06.636897 (+ 62us) server_negotiation.cc:407] Connection header received
0808 14:43:06.637582 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:06.637599 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:06.637975 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:06.638324 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:06.639190 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:06.639194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:06.639207 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:06.639219 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:06.639278 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:06.639298 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:06.639462 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:06.639534 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 14:43:06.639696 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":150,"threads_started":1}
I20260808 14:43:06.642150 8758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75: Bootstrap starting.
I20260808 14:43:06.643484 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:06.644804 8758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:06.645653 8758 log.cc:824] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:06.647553 8761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:06.645612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37943 (local address 127.0.0.1:45101)
0808 14:43:06.645767 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:06.645771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:06.646041 (+ 270us) server_negotiation.cc:407] Connection header received
0808 14:43:06.646182 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:06.646186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:06.646229 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:06.646318 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:06.647116 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:06.647119 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:06.647121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:06.647133 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:06.647161 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:06.647171 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:06.647251 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:06.647369 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 14:43:06.647405 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 14:43:06.648162 8758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75: No bootstrap required, opened a new log
I20260808 14:43:06.651880 8758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdf40211fad43bc80c2e64fdd680a75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45101 } }
I20260808 14:43:06.652204 8758 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:06.652297 8758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bdf40211fad43bc80c2e64fdd680a75, State: Initialized, Role: FOLLOWER
I20260808 14:43:06.652773 8758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdf40211fad43bc80c2e64fdd680a75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45101 } }
I20260808 14:43:06.652936 8758 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:06.653034 8758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:06.653141 8758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:06.654304 8758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdf40211fad43bc80c2e64fdd680a75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45101 } }
I20260808 14:43:06.654681 8758 leader_election.cc:302] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bdf40211fad43bc80c2e64fdd680a75; no voters:
I20260808 14:43:06.655030 8758 leader_election.cc:288] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:06.655185 8768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:06.655443 8768 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [term 1 LEADER]: Becoming Leader. State: Replica: 6bdf40211fad43bc80c2e64fdd680a75, State: Running, Role: LEADER
I20260808 14:43:06.655821 8768 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdf40211fad43bc80c2e64fdd680a75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45101 } }
I20260808 14:43:06.656421 8758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:06.657593 8771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bdf40211fad43bc80c2e64fdd680a75. Latest consensus state: current_term: 1 leader_uuid: "6bdf40211fad43bc80c2e64fdd680a75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdf40211fad43bc80c2e64fdd680a75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45101 } } }
I20260808 14:43:06.657759 8771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:06.658138 8778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:06.658100 8769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bdf40211fad43bc80c2e64fdd680a75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bdf40211fad43bc80c2e64fdd680a75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45101 } } }
I20260808 14:43:06.658234 8769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:06.661844 8778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:06.667827 8778 catalog_manager.cc:1347] Generated new cluster ID: 2d82326dcd1442179a07e712abb13229
I20260808 14:43:06.667919 8778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:06.681906 8778 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:06.682325 8778 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:06.688755 8778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bdf40211fad43bc80c2e64fdd680a75: Generated new TSK 0
I20260808 14:43:06.689559 8778 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:06.698779 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 8689
2026-08-08T14:43:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:06 dist-test-slave-p3t5 krb5kdc[8790](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:06 dist-test-slave-p3t5 krb5kdc[8790](info): set up 2 sockets
Aug 08 14:43:06 dist-test-slave-p3t5 krb5kdc[8790](info): commencing operation
krb5kdc: starting...
W20260808 14:43:07.763674 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:07 dist-test-slave-p3t5 krb5kdc[8790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:43:07.905857 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34259
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:08.027513 8803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:08.027846 8803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:08.027892 8803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:08.032166 8803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:08.032250 8803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:08.032271 8803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:08.032289 8803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:08.032305 8803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:08.038224 8803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:08.039439 8803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:08.040570 8803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:08.049800 8808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:08.049800 8809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:08.049980 8803 server_base.cc:1034] running on GCE node
W20260808 14:43:08.049886 8811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:08.050945 8803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:08.052357 8803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:08.053545 8803 hybrid_clock.cc:648] HybridClock initialized: now 1786200188053516 us; error 42 us; skew 500 ppm
Aug 08 14:43:08 dist-test-slave-p3t5 krb5kdc[8790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200188, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:08.056452 8803 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:43:08.057472 8803 webserver.cc:460] Webserver started at http://127.0.0.1:41691/ using document root <none> and password file <none>
I20260808 14:43:08.058136 8803 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:08.058245 8803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:08.058508 8803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:08.060638 8803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "fc57f203eaf94ab1aed7afa9fd7afb0f"
format_stamp: "Formatted at 2026-08-08 14:43:08 on dist-test-slave-p3t5"
server_key: "8a8f361c1c40cd271d3865b0d92dde9e"
server_key_iv: "79d5482e6540e7b906fa29fa2dc1f4f3"
server_key_version: "encryptionkey@0"
I20260808 14:43:08.061249 8803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "fc57f203eaf94ab1aed7afa9fd7afb0f"
format_stamp: "Formatted at 2026-08-08 14:43:08 on dist-test-slave-p3t5"
server_key: "8a8f361c1c40cd271d3865b0d92dde9e"
server_key_iv: "79d5482e6540e7b906fa29fa2dc1f4f3"
server_key_version: "encryptionkey@0"
I20260808 14:43:08.065140 8803 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:43:08.067812 8819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:08.069128 8803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 14:43:08.069280 8803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "fc57f203eaf94ab1aed7afa9fd7afb0f"
format_stamp: "Formatted at 2026-08-08 14:43:08 on dist-test-slave-p3t5"
server_key: "8a8f361c1c40cd271d3865b0d92dde9e"
server_key_iv: "79d5482e6540e7b906fa29fa2dc1f4f3"
server_key_version: "encryptionkey@0"
I20260808 14:43:08.069398 8803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:08.093591 8803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:08.096858 8803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:08.097098 8803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:08.105105 8803 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34259
I20260808 14:43:08.105129 8875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34259 every 8 connection(s)
I20260808 14:43:08.106227 8803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:08.109222 8876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:08.114044 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 8803
I20260808 14:43:08.114163 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:08.114452 20033 external_mini_cluster.cc:1428] Setting key a0a51c36366ae70d37124f9af307f4b4
I20260808 14:43:08.115731 8876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f: Bootstrap starting.
I20260808 14:43:08.118223 8876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:08.119029 8876 log.cc:824] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:08.121022 8876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f: No bootstrap required, opened a new log
Aug 08 14:43:08 dist-test-slave-p3t5 krb5kdc[8790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 14:43:08.125618 8876 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc57f203eaf94ab1aed7afa9fd7afb0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34259 } }
I20260808 14:43:08.125972 8876 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:08.126062 8876 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc57f203eaf94ab1aed7afa9fd7afb0f, State: Initialized, Role: FOLLOWER
I20260808 14:43:08.126559 8876 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc57f203eaf94ab1aed7afa9fd7afb0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34259 } }
I20260808 14:43:08.126690 8876 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:08.126818 8876 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:08.126963 8876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:08.128159 8876 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc57f203eaf94ab1aed7afa9fd7afb0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34259 } }
I20260808 14:43:08.128571 8876 leader_election.cc:302] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc57f203eaf94ab1aed7afa9fd7afb0f; no voters:
I20260808 14:43:08.128993 8876 leader_election.cc:288] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:08.129101 8881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:08.129388 8881 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [term 1 LEADER]: Becoming Leader. State: Replica: fc57f203eaf94ab1aed7afa9fd7afb0f, State: Running, Role: LEADER
I20260808 14:43:08.129812 8881 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc57f203eaf94ab1aed7afa9fd7afb0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34259 } }
I20260808 14:43:08.130360 8879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:08.115984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40363 (local address 127.0.0.1:34259)
0808 14:43:08.116448 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:08.116471 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:08.116509 (+ 38us) server_negotiation.cc:407] Connection header received
0808 14:43:08.117175 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:08.117192 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:08.117569 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:08.117989 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:08.118856 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:08.119957 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:08.120912 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:08.121446 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:08.121694 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:08.124689 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:08.124695 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:08.124712 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:08.124743 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:08.126827 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:08.127392 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:08.127397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:08.127402 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:08.127537 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:08.127836 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:08.127838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:08.127839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:08.128368 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:08.128538 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:08.128861 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 14:43:08.129247 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":148,"threads_started":1}
I20260808 14:43:08.130475 8876 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:08.131567 8882 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc57f203eaf94ab1aed7afa9fd7afb0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc57f203eaf94ab1aed7afa9fd7afb0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34259 } } }
I20260808 14:43:08.131697 8882 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:08.131930 8882 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc57f203eaf94ab1aed7afa9fd7afb0f. Latest consensus state: current_term: 1 leader_uuid: "fc57f203eaf94ab1aed7afa9fd7afb0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc57f203eaf94ab1aed7afa9fd7afb0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34259 } } }
I20260808 14:43:08.132014 8882 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:08.132395 8890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:08.134023 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:08.136297 8890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:08.143586 8890 catalog_manager.cc:1347] Generated new cluster ID: c5a96c82da2e4da2932644de2b0e3b23
I20260808 14:43:08.143684 8890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:08.147006 8879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:08.136453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58635 (local address 127.0.0.1:34259)
0808 14:43:08.136582 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:08.136597 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:08.137966 (+ 1369us) server_negotiation.cc:407] Connection header received
0808 14:43:08.138116 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:08.138119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:08.138179 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:08.138296 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:08.139007 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:08.139528 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:08.140364 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:08.140675 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:08.140788 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:08.143568 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:08.143572 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:08.143574 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:08.143613 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:08.145205 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:08.145671 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:08.145675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:08.145677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:08.145773 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:08.146081 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:08.146084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:08.146085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:08.146265 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:08.146400 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:08.146649 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 14:43:08.146804 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 14:43:08.159696 8890 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:08.161033 8890 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:08.168108 8890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc57f203eaf94ab1aed7afa9fd7afb0f: Generated new TSK 0
I20260808 14:43:08.169047 8890 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:08.174950 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 8803
2026-08-08T14:43:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1476 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:08 dist-test-slave-p3t5 krb5kdc[8908](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:08 dist-test-slave-p3t5 krb5kdc[8908](info): set up 2 sockets
Aug 08 14:43:08 dist-test-slave-p3t5 krb5kdc[8908](info): commencing operation
krb5kdc: starting...
W20260808 14:43:09.251363 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:09 dist-test-slave-p3t5 krb5kdc[8908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:43:09.392967 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33209
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:09.515486 8921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:09.515865 8921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:09.515964 8921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:09.520252 8921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:09.520364 8921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:09.520401 8921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:09.520443 8921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:09.520476 8921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:09.526204 8921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:09.527456 8921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:09.528584 8921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:09.537990 8929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:09.537974 8927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:09.538012 8926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:09.538169 8921 server_base.cc:1034] running on GCE node
I20260808 14:43:09.538954 8921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:09.540310 8921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:09.541524 8921 hybrid_clock.cc:648] HybridClock initialized: now 1786200189541510 us; error 32 us; skew 500 ppm
Aug 08 14:43:09 dist-test-slave-p3t5 krb5kdc[8908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200189, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:09.545166 8921 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:43:09.546356 8921 webserver.cc:460] Webserver started at http://127.0.0.1:46183/ using document root <none> and password file <none>
I20260808 14:43:09.547009 8921 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:09.547138 8921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:09.547420 8921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:09.549747 8921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "d031ef399e42405490778153cbcafc4f"
format_stamp: "Formatted at 2026-08-08 14:43:09 on dist-test-slave-p3t5"
server_key: "3c9130032fc53566b040516a9a680fd0"
server_key_iv: "789e1e4031386766be5e85dafe04a92d"
server_key_version: "encryptionkey@0"
I20260808 14:43:09.550442 8921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "d031ef399e42405490778153cbcafc4f"
format_stamp: "Formatted at 2026-08-08 14:43:09 on dist-test-slave-p3t5"
server_key: "3c9130032fc53566b040516a9a680fd0"
server_key_iv: "789e1e4031386766be5e85dafe04a92d"
server_key_version: "encryptionkey@0"
I20260808 14:43:09.554631 8921 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:43:09.557391 8937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:09.558950 8921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:43:09.559146 8921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "d031ef399e42405490778153cbcafc4f"
format_stamp: "Formatted at 2026-08-08 14:43:09 on dist-test-slave-p3t5"
server_key: "3c9130032fc53566b040516a9a680fd0"
server_key_iv: "789e1e4031386766be5e85dafe04a92d"
server_key_version: "encryptionkey@0"
I20260808 14:43:09.559298 8921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:09.582346 8921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:09.586304 8921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:09.586556 8921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:09.594645 8921 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33209
I20260808 14:43:09.594674 8990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33209 every 8 connection(s)
I20260808 14:43:09.596113 8921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:09.599416 8991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:09.601392 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 8921
I20260808 14:43:09.601550 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:09.601960 20033 external_mini_cluster.cc:1428] Setting key 16bb1a2905ef1f4c9a6a7b40b04225fa
I20260808 14:43:09.606997 8991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f: Bootstrap starting.
I20260808 14:43:09.609915 8991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:09.610814 8991 log.cc:824] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:09.612741 8994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:09.603508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40207 (local address 127.0.0.1:33209)
0808 14:43:09.603975 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:09.603995 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:09.604040 (+ 45us) server_negotiation.cc:407] Connection header received
0808 14:43:09.604918 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:09.604947 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:09.605418 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:09.605839 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:09.606274 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:09.607721 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:09.608713 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:09.609421 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:09.609590 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:09.610629 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:09.610637 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:09.610657 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:09.611205 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:09.611288 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:09.611323 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:09.611455 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:09.611610 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 14:43:09.611978 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":113,"threads_started":1}
I20260808 14:43:09.613075 8991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f: No bootstrap required, opened a new log
I20260808 14:43:09.616083 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:09.617698 8991 raft_consensus.cc:348] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d031ef399e42405490778153cbcafc4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33209 } }
I20260808 14:43:09.618010 8991 raft_consensus.cc:374] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:09.618077 8991 raft_consensus.cc:729] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d031ef399e42405490778153cbcafc4f, State: Initialized, Role: FOLLOWER
I20260808 14:43:09.618629 8991 consensus_queue.cc:260] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d031ef399e42405490778153cbcafc4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33209 } }
I20260808 14:43:09.618819 8991 raft_consensus.cc:388] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:09.618899 8991 raft_consensus.cc:482] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:09.619019 8991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:09.620337 8991 raft_consensus.cc:504] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d031ef399e42405490778153cbcafc4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33209 } }
I20260808 14:43:09.620718 8991 leader_election.cc:302] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d031ef399e42405490778153cbcafc4f; no voters:
I20260808 14:43:09.621093 8991 leader_election.cc:288] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:09.621240 9002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:09.621506 9002 raft_consensus.cc:686] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [term 1 LEADER]: Becoming Leader. State: Replica: d031ef399e42405490778153cbcafc4f, State: Running, Role: LEADER
I20260808 14:43:09.621896 9002 consensus_queue.cc:237] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d031ef399e42405490778153cbcafc4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33209 } }
I20260808 14:43:09.623020 8991 sys_catalog.cc:564] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:09.623184 8994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:09.618378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55753 (local address 127.0.0.1:33209)
0808 14:43:09.618490 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:09.618494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:09.618700 (+ 206us) server_negotiation.cc:407] Connection header received
0808 14:43:09.618804 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:09.618808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:09.618866 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:09.618961 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:09.619290 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:09.620074 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:09.621010 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:09.621405 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:09.621967 (+ 562us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:09.622466 (+ 499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:09.622470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:09.622472 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:09.622486 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:09.622515 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:09.622528 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:09.622903 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:09.622938 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 14:43:09.622990 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 14:43:09.623643 9003 sys_catalog.cc:455] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d031ef399e42405490778153cbcafc4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d031ef399e42405490778153cbcafc4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33209 } } }
I20260808 14:43:09.623772 9003 sys_catalog.cc:458] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:09.624155 9004 sys_catalog.cc:455] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d031ef399e42405490778153cbcafc4f. Latest consensus state: current_term: 1 leader_uuid: "d031ef399e42405490778153cbcafc4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d031ef399e42405490778153cbcafc4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33209 } } }
I20260808 14:43:09.624318 9004 sys_catalog.cc:458] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:09.624639 9007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:09.628062 9007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:09.634351 9007 catalog_manager.cc:1347] Generated new cluster ID: a2486b2a7da74e529386514aa00ac6df
I20260808 14:43:09.634441 9007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:09.645917 9007 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:09.647356 9007 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:09.670136 9007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d031ef399e42405490778153cbcafc4f: Generated new TSK 0
I20260808 14:43:09.671113 9007 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:09.682449 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 8921
2026-08-08T14:43:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1507 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:09 dist-test-slave-p3t5 krb5kdc[9025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:09 dist-test-slave-p3t5 krb5kdc[9025](info): set up 2 sockets
Aug 08 14:43:09 dist-test-slave-p3t5 krb5kdc[9025](info): commencing operation
krb5kdc: starting...
W20260808 14:43:10.750082 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:10 dist-test-slave-p3t5 krb5kdc[9025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:43:10.892628 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45871
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:11.011878 9038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:11.012255 9038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:11.012331 9038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:11.016569 9038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:11.016698 9038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:11.016736 9038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:11.016772 9038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:11.016809 9038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:11.022970 9038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:11.024230 9038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:11.025373 9038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:11.034550 9047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:11.034552 9043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:11.034552 9044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:11.035012 9038 server_base.cc:1034] running on GCE node
I20260808 14:43:11.035607 9038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:11.036715 9038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:11.037894 9038 hybrid_clock.cc:648] HybridClock initialized: now 1786200191037860 us; error 43 us; skew 500 ppm
Aug 08 14:43:11 dist-test-slave-p3t5 krb5kdc[9025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200191, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:11.040841 9038 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:43:11.041915 9038 webserver.cc:460] Webserver started at http://127.0.0.1:45387/ using document root <none> and password file <none>
I20260808 14:43:11.042531 9038 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:11.042630 9038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:11.042892 9038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:11.045078 9038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "43892714649d4851b4e86087e56588d8"
format_stamp: "Formatted at 2026-08-08 14:43:11 on dist-test-slave-p3t5"
server_key: "d496837f80220eaf86b09bc7be8b637c"
server_key_iv: "ea5fe990ff9f02be2a97b05f47f0aa4e"
server_key_version: "encryptionkey@0"
I20260808 14:43:11.045747 9038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "43892714649d4851b4e86087e56588d8"
format_stamp: "Formatted at 2026-08-08 14:43:11 on dist-test-slave-p3t5"
server_key: "d496837f80220eaf86b09bc7be8b637c"
server_key_iv: "ea5fe990ff9f02be2a97b05f47f0aa4e"
server_key_version: "encryptionkey@0"
I20260808 14:43:11.049777 9038 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:43:11.052461 9054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:11.053908 9038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:43:11.054080 9038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "43892714649d4851b4e86087e56588d8"
format_stamp: "Formatted at 2026-08-08 14:43:11 on dist-test-slave-p3t5"
server_key: "d496837f80220eaf86b09bc7be8b637c"
server_key_iv: "ea5fe990ff9f02be2a97b05f47f0aa4e"
server_key_version: "encryptionkey@0"
I20260808 14:43:11.054215 9038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:11.059077 9038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:11.062264 9038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:11.062474 9038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:11.069552 9038 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45871
I20260808 14:43:11.069561 9108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45871 every 8 connection(s)
I20260808 14:43:11.070586 9038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:11.070907 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 9038
I20260808 14:43:11.071041 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:11.071374 20033 external_mini_cluster.cc:1428] Setting key febca955aa082485ac9ab1ed94a14956
I20260808 14:43:11.074107 9109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:11.076941 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:11.072775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34887 (local address 127.0.0.1:45871)
0808 14:43:11.073213 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:11.073230 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:11.073268 (+ 38us) server_negotiation.cc:407] Connection header received
0808 14:43:11.074024 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:11.074045 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:11.074401 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:11.074710 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:11.075634 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:11.075639 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:11.075652 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:11.075663 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:11.075719 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:11.075740 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:11.075855 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:11.075989 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 14:43:11.076173 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":151,"threads_started":1}
I20260808 14:43:11.080129 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:11.081008 9109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8: Bootstrap starting.
I20260808 14:43:11.084035 9109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:11.084307 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:11.082517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58533 (local address 127.0.0.1:45871)
0808 14:43:11.082651 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:11.082656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:11.082906 (+ 250us) server_negotiation.cc:407] Connection header received
0808 14:43:11.083021 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:11.083023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:11.083058 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:11.083125 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:11.083838 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:11.083841 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:11.083843 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:11.083854 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:11.083880 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:11.083890 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:11.083966 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:11.084117 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 14:43:11.084167 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 14:43:11.085011 9109 log.cc:824] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:11.087134 9109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8: No bootstrap required, opened a new log
I20260808 14:43:11.090768 9109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43892714649d4851b4e86087e56588d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45871 } }
I20260808 14:43:11.091022 9109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:11.091100 9109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43892714649d4851b4e86087e56588d8, State: Initialized, Role: FOLLOWER
I20260808 14:43:11.091616 9109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43892714649d4851b4e86087e56588d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45871 } }
I20260808 14:43:11.091771 9109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:11.091861 9109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:11.091988 9109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:11.092954 9109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43892714649d4851b4e86087e56588d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45871 } }
I20260808 14:43:11.093321 9109 leader_election.cc:302] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43892714649d4851b4e86087e56588d8; no voters:
I20260808 14:43:11.093725 9109 leader_election.cc:288] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:11.093858 9120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:11.094120 9120 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [term 1 LEADER]: Becoming Leader. State: Replica: 43892714649d4851b4e86087e56588d8, State: Running, Role: LEADER
I20260808 14:43:11.094549 9120 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43892714649d4851b4e86087e56588d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45871 } }
I20260808 14:43:11.095196 9109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:11.096462 9122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43892714649d4851b4e86087e56588d8. Latest consensus state: current_term: 1 leader_uuid: "43892714649d4851b4e86087e56588d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43892714649d4851b4e86087e56588d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45871 } } }
I20260808 14:43:11.096483 9121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43892714649d4851b4e86087e56588d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43892714649d4851b4e86087e56588d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45871 } } }
I20260808 14:43:11.096603 9122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:11.096608 9121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:11.097035 9129 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:11.099929 9129 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:11.105942 9129 catalog_manager.cc:1347] Generated new cluster ID: 6f7dfe3a94a749ccb7ef9e8f1acde275
I20260808 14:43:11.106053 9129 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:11.117105 9129 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:11.117496 9129 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:11.124251 9129 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43892714649d4851b4e86087e56588d8: Generated new TSK 0
I20260808 14:43:11.125097 9129 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:11.133253 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 9038
2026-08-08T14:43:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1449 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:11 dist-test-slave-p3t5 krb5kdc[9141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:11 dist-test-slave-p3t5 krb5kdc[9141](info): set up 2 sockets
Aug 08 14:43:11 dist-test-slave-p3t5 krb5kdc[9141](info): commencing operation
krb5kdc: starting...
W20260808 14:43:12.197283 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:12 dist-test-slave-p3t5 krb5kdc[9141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:43:12.335134 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41559
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:12.453848 9154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:12.454214 9154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:12.454301 9154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:12.458382 9154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:12.458490 9154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:12.458532 9154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:12.458593 9154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:12.458635 9154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:12.464160 9154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:12.465353 9154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:12.466542 9154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:12.475796 9163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:12.475853 9161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:12.475868 9160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:12.476224 9154 server_base.cc:1034] running on GCE node
I20260808 14:43:12.476933 9154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:12.478296 9154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:12.479511 9154 hybrid_clock.cc:648] HybridClock initialized: now 1786200192479491 us; error 35 us; skew 500 ppm
Aug 08 14:43:12 dist-test-slave-p3t5 krb5kdc[9141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200192, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:12.488081 9154 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:43:12.489212 9154 webserver.cc:460] Webserver started at http://127.0.0.1:44609/ using document root <none> and password file <none>
I20260808 14:43:12.489888 9154 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:12.489972 9154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:12.490162 9154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:12.493217 9154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "2cc4b14f2aa949ee99410f4edfbcdf1e"
format_stamp: "Formatted at 2026-08-08 14:43:12 on dist-test-slave-p3t5"
server_key: "ab42c55d7bdfde17c4edef272b64229e"
server_key_iv: "7f036bf689ba4c9be79cf9c14ec5f260"
server_key_version: "encryptionkey@0"
I20260808 14:43:12.493975 9154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "2cc4b14f2aa949ee99410f4edfbcdf1e"
format_stamp: "Formatted at 2026-08-08 14:43:12 on dist-test-slave-p3t5"
server_key: "ab42c55d7bdfde17c4edef272b64229e"
server_key_iv: "7f036bf689ba4c9be79cf9c14ec5f260"
server_key_version: "encryptionkey@0"
I20260808 14:43:12.498374 9154 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:43:12.500962 9170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:12.502367 9154 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:43:12.502530 9154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "2cc4b14f2aa949ee99410f4edfbcdf1e"
format_stamp: "Formatted at 2026-08-08 14:43:12 on dist-test-slave-p3t5"
server_key: "ab42c55d7bdfde17c4edef272b64229e"
server_key_iv: "7f036bf689ba4c9be79cf9c14ec5f260"
server_key_version: "encryptionkey@0"
I20260808 14:43:12.502660 9154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:12.524356 9154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:12.528053 9154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:12.528290 9154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:12.536161 9154 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41559
I20260808 14:43:12.536188 9223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41559 every 8 connection(s)
I20260808 14:43:12.537223 9154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:12.540131 9224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:12.543263 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 9154
I20260808 14:43:12.543390 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:12.543692 20033 external_mini_cluster.cc:1428] Setting key 8168ef7751f5f43deec7c50d014e08b4
I20260808 14:43:12.546298 9224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e: Bootstrap starting.
I20260808 14:43:12.548599 9224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:12.549355 9224 log.cc:824] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:12.551371 9224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e: No bootstrap required, opened a new log
Aug 08 14:43:12 dist-test-slave-p3t5 krb5kdc[9141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 14:43:12.555150 9224 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc4b14f2aa949ee99410f4edfbcdf1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41559 } }
I20260808 14:43:12.555433 9224 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:12.555518 9224 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc4b14f2aa949ee99410f4edfbcdf1e, State: Initialized, Role: FOLLOWER
I20260808 14:43:12.556072 9224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc4b14f2aa949ee99410f4edfbcdf1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41559 } }
I20260808 14:43:12.556250 9224 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:12.556330 9224 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:12.556439 9224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:12.557464 9224 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc4b14f2aa949ee99410f4edfbcdf1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41559 } }
I20260808 14:43:12.557946 9224 leader_election.cc:302] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cc4b14f2aa949ee99410f4edfbcdf1e; no voters:
I20260808 14:43:12.558413 9224 leader_election.cc:288] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:12.558522 9229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:12.558830 9229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [term 1 LEADER]: Becoming Leader. State: Replica: 2cc4b14f2aa949ee99410f4edfbcdf1e, State: Running, Role: LEADER
I20260808 14:43:12.559218 9229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc4b14f2aa949ee99410f4edfbcdf1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41559 } }
I20260808 14:43:12.559811 9227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:12.545276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:34711 (local address 127.0.0.1:41559)
0808 14:43:12.545803 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:12.545821 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:12.545860 (+ 39us) server_negotiation.cc:407] Connection header received
0808 14:43:12.546564 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:12.546584 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:12.546974 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:12.547333 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:12.548209 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:12.549303 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:12.550554 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:12.551042 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:12.551226 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:12.553982 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:12.553988 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:12.554003 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:12.554035 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:12.556061 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:12.556640 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:12.556645 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:12.556650 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:12.556773 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:12.557153 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:12.557157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:12.557158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:12.557824 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:12.558130 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:12.558472 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 14:43:12.558745 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":153,"threads_started":1}
I20260808 14:43:12.560045 9224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:12.563683 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:12.563910 9230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cc4b14f2aa949ee99410f4edfbcdf1e. Latest consensus state: current_term: 1 leader_uuid: "2cc4b14f2aa949ee99410f4edfbcdf1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc4b14f2aa949ee99410f4edfbcdf1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41559 } } }
I20260808 14:43:12.564056 9230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [sys.catalog]: This master's current role is: LEADER
W20260808 14:43:12.565933 9243 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:43:12.566028 9243 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:43:12.566162 9231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cc4b14f2aa949ee99410f4edfbcdf1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc4b14f2aa949ee99410f4edfbcdf1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41559 } } }
I20260808 14:43:12.566498 9231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:12.566978 9246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:12.568044 9246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:12.575899 9246 catalog_manager.cc:1347] Generated new cluster ID: 5f0f623a73094376aa5aa83037469f9e
I20260808 14:43:12.575981 9246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:12.576944 9227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:12.565973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:48955 (local address 127.0.0.1:41559)
0808 14:43:12.566086 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:12.566091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:12.566620 (+ 529us) server_negotiation.cc:407] Connection header received
0808 14:43:12.567261 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:12.567279 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:12.567333 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:12.567436 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:12.568286 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:12.569051 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:12.570292 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:12.570733 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:12.570833 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:12.572874 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:12.572878 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:12.572880 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:12.572908 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:12.574753 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:12.575374 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:12.575379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:12.575380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:12.575487 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:12.575900 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:12.575904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:12.575905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:12.576164 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:12.576280 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:12.576680 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 14:43:12.576782 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 14:43:12.605183 9246 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:12.606520 9246 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:12.614583 9246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cc4b14f2aa949ee99410f4edfbcdf1e: Generated new TSK 0
I20260808 14:43:12.615420 9246 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:12.622850 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 9154
W20260808 14:43:12.631249 9139 connection.cc:535] client connection to 127.0.0.1:41559 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41559): Connection reset by peer (error 104)
2026-08-08T14:43:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1490 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:12 dist-test-slave-p3t5 krb5kdc[9255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:12 dist-test-slave-p3t5 krb5kdc[9255](info): set up 2 sockets
Aug 08 14:43:12 dist-test-slave-p3t5 krb5kdc[9255](info): commencing operation
krb5kdc: starting...
W20260808 14:43:13.719540 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:13 dist-test-slave-p3t5 krb5kdc[9255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:43:13.857115 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42075
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:13.977082 9268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:13.977432 9268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:13.977512 9268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:13.981746 9268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:13.981842 9268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:13.981881 9268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:13.981928 9268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:13.981961 9268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:13.987465 9268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:13.988659 9268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:13.989782 9268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:13.998790 9273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:13.998966 9268 server_base.cc:1034] running on GCE node
W20260808 14:43:13.998799 9274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:13.998790 9276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:13.999693 9268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:14.000886 9268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:14.002118 9268 hybrid_clock.cc:648] HybridClock initialized: now 1786200194002101 us; error 30 us; skew 500 ppm
Aug 08 14:43:14 dist-test-slave-p3t5 krb5kdc[9255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200194, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:14.005501 9268 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:43:14.006590 9268 webserver.cc:460] Webserver started at http://127.0.0.1:39131/ using document root <none> and password file <none>
I20260808 14:43:14.007210 9268 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:14.007311 9268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:14.007563 9268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:14.009845 9268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "f1de54ccce584f939789bcb9618b0fe3"
format_stamp: "Formatted at 2026-08-08 14:43:14 on dist-test-slave-p3t5"
server_key: "7ccd3bce507f1cf8f4872c4d8522f204"
server_key_iv: "01747712494f9fc52e74fc8a91e2ce0e"
server_key_version: "encryptionkey@0"
I20260808 14:43:14.010463 9268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "f1de54ccce584f939789bcb9618b0fe3"
format_stamp: "Formatted at 2026-08-08 14:43:14 on dist-test-slave-p3t5"
server_key: "7ccd3bce507f1cf8f4872c4d8522f204"
server_key_iv: "01747712494f9fc52e74fc8a91e2ce0e"
server_key_version: "encryptionkey@0"
I20260808 14:43:14.014683 9268 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:43:14.017356 9283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:14.018765 9268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:43:14.018944 9268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "f1de54ccce584f939789bcb9618b0fe3"
format_stamp: "Formatted at 2026-08-08 14:43:14 on dist-test-slave-p3t5"
server_key: "7ccd3bce507f1cf8f4872c4d8522f204"
server_key_iv: "01747712494f9fc52e74fc8a91e2ce0e"
server_key_version: "encryptionkey@0"
I20260808 14:43:14.019079 9268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:14.035113 9268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:14.038841 9268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:14.039083 9268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:14.046622 9268 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42075
I20260808 14:43:14.046649 9337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42075 every 8 connection(s)
I20260808 14:43:14.047724 9268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:14.050737 9338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:14.055202 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 9268
I20260808 14:43:14.055322 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:14.055630 20033 external_mini_cluster.cc:1428] Setting key 56e711e47a5536d2dead0667af08d82e
I20260808 14:43:14.056824 9338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3: Bootstrap starting.
I20260808 14:43:14.059311 9338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:14.060079 9338 log.cc:824] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:14.062143 9338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3: No bootstrap required, opened a new log
I20260808 14:43:14.065001 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:14.057120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:52299 (local address 127.0.0.1:42075)
0808 14:43:14.057580 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:14.057596 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:14.057683 (+ 87us) server_negotiation.cc:407] Connection header received
0808 14:43:14.058351 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:14.058370 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:14.058727 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:14.059084 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:14.059552 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:14.060618 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:14.061541 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:14.062096 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:14.062283 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:14.063250 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:14.063256 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:14.063270 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:14.063674 (+ 404us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:14.063730 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:14.063753 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:14.063896 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:14.064092 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 14:43:14.064242 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":144,"threads_started":1}
I20260808 14:43:14.066159 9338 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1de54ccce584f939789bcb9618b0fe3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42075 } }
I20260808 14:43:14.066445 9338 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:14.066540 9338 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1de54ccce584f939789bcb9618b0fe3, State: Initialized, Role: FOLLOWER
I20260808 14:43:14.067006 9338 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1de54ccce584f939789bcb9618b0fe3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42075 } }
I20260808 14:43:14.067185 9338 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:14.067260 9338 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:14.067363 9338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:14.068454 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:14.068509 9338 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1de54ccce584f939789bcb9618b0fe3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42075 } }
I20260808 14:43:14.068895 9338 leader_election.cc:302] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1de54ccce584f939789bcb9618b0fe3; no voters:
I20260808 14:43:14.069231 9338 leader_election.cc:288] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:14.069339 9343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:14.069581 9343 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [term 1 LEADER]: Becoming Leader. State: Replica: f1de54ccce584f939789bcb9618b0fe3, State: Running, Role: LEADER
I20260808 14:43:14.070048 9343 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1de54ccce584f939789bcb9618b0fe3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42075 } }
I20260808 14:43:14.070749 9338 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:14.071350 9345 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1de54ccce584f939789bcb9618b0fe3. Latest consensus state: current_term: 1 leader_uuid: "f1de54ccce584f939789bcb9618b0fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1de54ccce584f939789bcb9618b0fe3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42075 } } }
I20260808 14:43:14.071573 9345 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:14.071764 9344 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1de54ccce584f939789bcb9618b0fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1de54ccce584f939789bcb9618b0fe3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42075 } } }
I20260808 14:43:14.071854 9344 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:14.072305 9355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:14.077646 9355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:14.082417 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:14.072221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:38237 (local address 127.0.0.1:42075)
0808 14:43:14.072438 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:14.072443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:14.072458 (+ 15us) server_negotiation.cc:407] Connection header received
0808 14:43:14.072495 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:14.072498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:14.072543 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:14.072619 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:14.073042 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:14.073802 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:14.077438 (+ 3636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:14.077829 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:14.077945 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:14.081885 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:14.081889 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:14.081892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:14.081908 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:14.081952 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:14.081964 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:14.082069 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:14.082226 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 14:43:14.082264 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260808 14:43:14.088907 9355 catalog_manager.cc:1347] Generated new cluster ID: bda760047db2462a9483ae7f95993165
I20260808 14:43:14.089082 9355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:14.095754 9355 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:14.097169 9355 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:14.103638 9355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1de54ccce584f939789bcb9618b0fe3: Generated new TSK 0
I20260808 14:43:14.104784 9355 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:14.114464 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 9268
W20260808 14:43:14.124802 9253 connection.cc:535] client connection to 127.0.0.1:42075 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42075): Connection reset by peer (error 104)
2026-08-08T14:43:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1493 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:14 dist-test-slave-p3t5 krb5kdc[9369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:14 dist-test-slave-p3t5 krb5kdc[9369](info): set up 2 sockets
Aug 08 14:43:14 dist-test-slave-p3t5 krb5kdc[9369](info): commencing operation
krb5kdc: starting...
W20260808 14:43:15.191574 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:15 dist-test-slave-p3t5 krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:43:15.341871 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39753
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:15.464208 9383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:15.464534 9383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:15.464579 9383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:15.468926 9383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:15.469022 9383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:15.469043 9383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:15.469059 9383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:15.469075 9383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:15.474872 9383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:15.476146 9383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:15.477317 9383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:15.486613 9388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:15.486989 9391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:15.487085 9389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:15.487155 9383 server_base.cc:1034] running on GCE node
I20260808 14:43:15.487975 9383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:15.489234 9383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:15.490417 9383 hybrid_clock.cc:648] HybridClock initialized: now 1786200195490357 us; error 76 us; skew 500 ppm
Aug 08 14:43:15 dist-test-slave-p3t5 krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:15.493430 9383 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:43:15.494997 9383 webserver.cc:460] Webserver started at http://127.0.0.1:44033/ using document root <none> and password file <none>
I20260808 14:43:15.495585 9383 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:15.495649 9383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:15.495823 9383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:15.498106 9383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "28f7e2af1fff4bd587f80e8185ca11f1"
format_stamp: "Formatted at 2026-08-08 14:43:15 on dist-test-slave-p3t5"
server_key: "9d887502d900c31ad25d58817553bca8"
server_key_iv: "e65c8c4e9d4a15b209891bafcdb52646"
server_key_version: "encryptionkey@0"
I20260808 14:43:15.498692 9383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "28f7e2af1fff4bd587f80e8185ca11f1"
format_stamp: "Formatted at 2026-08-08 14:43:15 on dist-test-slave-p3t5"
server_key: "9d887502d900c31ad25d58817553bca8"
server_key_iv: "e65c8c4e9d4a15b209891bafcdb52646"
server_key_version: "encryptionkey@0"
I20260808 14:43:15.502983 9383 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:43:15.506129 9398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:15.508208 9383 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260808 14:43:15.508391 9383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "28f7e2af1fff4bd587f80e8185ca11f1"
format_stamp: "Formatted at 2026-08-08 14:43:15 on dist-test-slave-p3t5"
server_key: "9d887502d900c31ad25d58817553bca8"
server_key_iv: "e65c8c4e9d4a15b209891bafcdb52646"
server_key_version: "encryptionkey@0"
I20260808 14:43:15.508555 9383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:15.514096 9383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:15.517845 9383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:15.518030 9383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:15.525826 9383 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39753
I20260808 14:43:15.525854 9452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39753 every 8 connection(s)
I20260808 14:43:15.526937 9383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:15.530099 9453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:15.530455 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 9383
I20260808 14:43:15.530596 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:15.530921 20033 external_mini_cluster.cc:1428] Setting key b7a25f28f32ae930f87772ab5f799682
I20260808 14:43:15.537295 9456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:15.532536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:44071 (local address 127.0.0.1:39753)
0808 14:43:15.533015 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:15.533035 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:15.533084 (+ 49us) server_negotiation.cc:407] Connection header received
0808 14:43:15.533937 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:15.533965 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:15.534380 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:15.534822 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:15.535710 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:15.535716 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:15.535733 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:15.535746 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:15.535825 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:15.535854 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:15.536015 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:15.536092 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 14:43:15.536271 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":119,"threads_started":1}
I20260808 14:43:15.538476 9453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1: Bootstrap starting.
I20260808 14:43:15.540777 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:15.541249 9453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:15.542128 9453 log.cc:824] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:15.544678 9453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1: No bootstrap required, opened a new log
I20260808 14:43:15.545217 9456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:15.543285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:36467 (local address 127.0.0.1:39753)
0808 14:43:15.543419 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:15.543426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:15.543625 (+ 199us) server_negotiation.cc:407] Connection header received
0808 14:43:15.543791 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:15.543794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:15.543831 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:15.543930 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:15.544712 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:15.544715 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:15.544718 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:15.544752 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:15.544786 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:15.544799 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:15.544886 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:15.545040 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 14:43:15.545087 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 14:43:15.548759 9453 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f7e2af1fff4bd587f80e8185ca11f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39753 } }
I20260808 14:43:15.549026 9453 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:15.549093 9453 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28f7e2af1fff4bd587f80e8185ca11f1, State: Initialized, Role: FOLLOWER
I20260808 14:43:15.549590 9453 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f7e2af1fff4bd587f80e8185ca11f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39753 } }
I20260808 14:43:15.549801 9453 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:15.549875 9453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:15.549993 9453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:15.551080 9453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f7e2af1fff4bd587f80e8185ca11f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39753 } }
I20260808 14:43:15.551539 9453 leader_election.cc:302] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28f7e2af1fff4bd587f80e8185ca11f1; no voters:
I20260808 14:43:15.551898 9453 leader_election.cc:288] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:15.552042 9463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:15.552315 9463 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [term 1 LEADER]: Becoming Leader. State: Replica: 28f7e2af1fff4bd587f80e8185ca11f1, State: Running, Role: LEADER
I20260808 14:43:15.552649 9463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f7e2af1fff4bd587f80e8185ca11f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39753 } }
I20260808 14:43:15.553311 9453 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:15.554627 9465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28f7e2af1fff4bd587f80e8185ca11f1. Latest consensus state: current_term: 1 leader_uuid: "28f7e2af1fff4bd587f80e8185ca11f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f7e2af1fff4bd587f80e8185ca11f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39753 } } }
I20260808 14:43:15.554627 9464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28f7e2af1fff4bd587f80e8185ca11f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f7e2af1fff4bd587f80e8185ca11f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39753 } } }
I20260808 14:43:15.554797 9465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:15.554797 9464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:15.555266 9473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:15.558593 9473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:15.565472 9473 catalog_manager.cc:1347] Generated new cluster ID: c4c1a39c28184f7f8a53dbae58d19b2e
I20260808 14:43:15.565568 9473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:15.581549 9473 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:15.581954 9473 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:15.590025 9473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28f7e2af1fff4bd587f80e8185ca11f1: Generated new TSK 0
I20260808 14:43:15.590961 9473 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:15.605014 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 9383
2026-08-08T14:43:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:15 dist-test-slave-p3t5 krb5kdc[9485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:15 dist-test-slave-p3t5 krb5kdc[9485](info): set up 2 sockets
Aug 08 14:43:15 dist-test-slave-p3t5 krb5kdc[9485](info): commencing operation
krb5kdc: starting...
W20260808 14:43:16.687633 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:16 dist-test-slave-p3t5 krb5kdc[9485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:43:16.830165 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42159
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:16.951184 9498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:16.951565 9498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:16.951663 9498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:16.955860 9498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:16.955981 9498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:16.956022 9498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:16.956060 9498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:16.956095 9498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:16.961766 9498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42159
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:16.963044 9498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:16.964206 9498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:16.974380 9504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:16.974371 9506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:16.974371 9503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:16.974593 9498 server_base.cc:1034] running on GCE node
I20260808 14:43:16.975531 9498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:16.976821 9498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:16.978130 9498 hybrid_clock.cc:648] HybridClock initialized: now 1786200196978090 us; error 62 us; skew 500 ppm
Aug 08 14:43:16 dist-test-slave-p3t5 krb5kdc[9485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:16.981550 9498 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:43:16.982719 9498 webserver.cc:460] Webserver started at http://127.0.0.1:40915/ using document root <none> and password file <none>
I20260808 14:43:16.983361 9498 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:16.983464 9498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:16.983723 9498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:16.986106 9498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "8d36a3fc45884a58ae0a79f0aa026c30"
format_stamp: "Formatted at 2026-08-08 14:43:16 on dist-test-slave-p3t5"
server_key: "af8146fe6007af6cbf8bc6c85d575a0f"
server_key_iv: "23238b2dd9db71508a99a0aa689ecc9d"
server_key_version: "encryptionkey@0"
I20260808 14:43:16.986765 9498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "8d36a3fc45884a58ae0a79f0aa026c30"
format_stamp: "Formatted at 2026-08-08 14:43:16 on dist-test-slave-p3t5"
server_key: "af8146fe6007af6cbf8bc6c85d575a0f"
server_key_iv: "23238b2dd9db71508a99a0aa689ecc9d"
server_key_version: "encryptionkey@0"
I20260808 14:43:16.990923 9498 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 14:43:16.993693 9513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:16.995064 9498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:43:16.995232 9498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "8d36a3fc45884a58ae0a79f0aa026c30"
format_stamp: "Formatted at 2026-08-08 14:43:16 on dist-test-slave-p3t5"
server_key: "af8146fe6007af6cbf8bc6c85d575a0f"
server_key_iv: "23238b2dd9db71508a99a0aa689ecc9d"
server_key_version: "encryptionkey@0"
I20260808 14:43:16.995369 9498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:17.007148 9498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:17.010892 9498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:17.011150 9498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:17.019042 9498 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42159
I20260808 14:43:17.019075 9567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42159 every 8 connection(s)
I20260808 14:43:17.020115 9498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:17.023365 9568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:17.028764 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 9498
I20260808 14:43:17.028992 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:17.029309 20033 external_mini_cluster.cc:1428] Setting key 85ab6cd44a2d854695a1ece2777d7025
I20260808 14:43:17.029738 9568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30: Bootstrap starting.
I20260808 14:43:17.032538 9568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:17.033352 9568 log.cc:824] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:17.035539 9568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30: No bootstrap required, opened a new log
Aug 08 14:43:17 dist-test-slave-p3t5 krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 14:43:17.039951 9568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d36a3fc45884a58ae0a79f0aa026c30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42159 } }
I20260808 14:43:17.040323 9568 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:17.040429 9568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d36a3fc45884a58ae0a79f0aa026c30, State: Initialized, Role: FOLLOWER
I20260808 14:43:17.041000 9568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d36a3fc45884a58ae0a79f0aa026c30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42159 } }
I20260808 14:43:17.041162 9568 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:17.041231 9568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:17.041348 9568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:17.042416 9568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d36a3fc45884a58ae0a79f0aa026c30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42159 } }
I20260808 14:43:17.042794 9568 leader_election.cc:302] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d36a3fc45884a58ae0a79f0aa026c30; no voters:
I20260808 14:43:17.043200 9568 leader_election.cc:288] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:17.043277 9573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:17.043486 9573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [term 1 LEADER]: Becoming Leader. State: Replica: 8d36a3fc45884a58ae0a79f0aa026c30, State: Running, Role: LEADER
I20260808 14:43:17.043820 9573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d36a3fc45884a58ae0a79f0aa026c30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42159 } }
I20260808 14:43:17.044648 9568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:17.045118 9573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d36a3fc45884a58ae0a79f0aa026c30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d36a3fc45884a58ae0a79f0aa026c30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42159 } } }
I20260808 14:43:17.045241 9573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:17.045476 9573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d36a3fc45884a58ae0a79f0aa026c30. Latest consensus state: current_term: 1 leader_uuid: "8d36a3fc45884a58ae0a79f0aa026c30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d36a3fc45884a58ae0a79f0aa026c30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42159 } } }
I20260808 14:43:17.045581 9573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:17.046100 9571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:17.030881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:49973 (local address 127.0.0.1:42159)
0808 14:43:17.031398 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:17.031417 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:17.031457 (+ 40us) server_negotiation.cc:407] Connection header received
0808 14:43:17.032150 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:17.032171 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:17.032572 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:17.032996 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:17.033979 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:17.035015 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:17.036055 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:17.036564 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:17.036765 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:17.039887 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:17.039894 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:17.039912 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:17.039952 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:17.042249 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:17.042904 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:17.042910 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:17.042915 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:17.043083 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:17.043529 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:17.043533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:17.043535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:17.044095 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:17.044342 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:17.044688 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 14:43:17.044979 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":166,"threads_started":1}
I20260808 14:43:17.046711 9577 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:17.049672 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:17.050230 9577 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:17.056596 9577 catalog_manager.cc:1347] Generated new cluster ID: 8e250a52842d4f98b66eb64fec956b2e
I20260808 14:43:17.056685 9577 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:17.063244 9571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:17.052169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:49535 (local address 127.0.0.1:42159)
0808 14:43:17.052303 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:17.052308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:17.052686 (+ 378us) server_negotiation.cc:407] Connection header received
0808 14:43:17.052906 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:17.052909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:17.052962 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:17.053016 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:17.054111 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:17.054951 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:17.056293 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:17.056660 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:17.056758 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:17.058996 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:17.059000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:17.059003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:17.059033 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:17.060813 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:17.061472 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:17.061476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:17.061478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:17.061576 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:17.062063 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:17.062067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:17.062069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:17.062325 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:17.062456 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:17.062941 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 14:43:17.063063 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 14:43:17.065375 9577 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:17.066531 9577 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:17.082129 9577 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d36a3fc45884a58ae0a79f0aa026c30: Generated new TSK 0
I20260808 14:43:17.083012 9577 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:17.087095 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 9498
W20260808 14:43:17.095417 9483 connection.cc:535] client connection to 127.0.0.1:42159 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42159): Connection reset by peer (error 104)
2026-08-08T14:43:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1482 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:17 dist-test-slave-p3t5 krb5kdc[9599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:17 dist-test-slave-p3t5 krb5kdc[9599](info): set up 2 sockets
Aug 08 14:43:17 dist-test-slave-p3t5 krb5kdc[9599](info): commencing operation
krb5kdc: starting...
W20260808 14:43:18.163280 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:18 dist-test-slave-p3t5 krb5kdc[9599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:43:18.308589 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37151
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:18.428980 9615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:18.429358 9615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:18.429447 9615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:18.433784 9615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:18.433950 9615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:18.433995 9615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:18.434051 9615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:18.434088 9615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:18.439882 9615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37151
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:18.441157 9615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:18.442337 9615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:18.451691 9621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:18.451788 9623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:18.451651 9620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:18.452117 9615 server_base.cc:1034] running on GCE node
I20260808 14:43:18.452728 9615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:18.454023 9615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:18.455224 9615 hybrid_clock.cc:648] HybridClock initialized: now 1786200198455191 us; error 46 us; skew 500 ppm
Aug 08 14:43:18 dist-test-slave-p3t5 krb5kdc[9599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200198, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:18.462049 9615 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:43:18.463100 9615 webserver.cc:460] Webserver started at http://127.0.0.1:37129/ using document root <none> and password file <none>
I20260808 14:43:18.463739 9615 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:18.463838 9615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:18.464162 9615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:18.466454 9615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "942737a005794d59a7e214d6369d77d2"
format_stamp: "Formatted at 2026-08-08 14:43:18 on dist-test-slave-p3t5"
server_key: "45a05d8b40c2c19a34850a90ed994075"
server_key_iv: "0ac9031dbc5db2402ae7dfb729046228"
server_key_version: "encryptionkey@0"
I20260808 14:43:18.467092 9615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "942737a005794d59a7e214d6369d77d2"
format_stamp: "Formatted at 2026-08-08 14:43:18 on dist-test-slave-p3t5"
server_key: "45a05d8b40c2c19a34850a90ed994075"
server_key_iv: "0ac9031dbc5db2402ae7dfb729046228"
server_key_version: "encryptionkey@0"
I20260808 14:43:18.471657 9615 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:43:18.474396 9630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:18.475840 9615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:43:18.476061 9615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "942737a005794d59a7e214d6369d77d2"
format_stamp: "Formatted at 2026-08-08 14:43:18 on dist-test-slave-p3t5"
server_key: "45a05d8b40c2c19a34850a90ed994075"
server_key_iv: "0ac9031dbc5db2402ae7dfb729046228"
server_key_version: "encryptionkey@0"
I20260808 14:43:18.476208 9615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:18.491735 9615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:18.495405 9615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:18.495648 9615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:18.503401 9615 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37151
I20260808 14:43:18.503428 9684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37151 every 8 connection(s)
I20260808 14:43:18.504475 9615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:18.507150 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 9615
I20260808 14:43:18.507310 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:18.507685 20033 external_mini_cluster.cc:1428] Setting key 6f8a77a16ae8ebb01eaf20bac7b36a5f
I20260808 14:43:18.507716 9685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:18.514739 9685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2: Bootstrap starting.
I20260808 14:43:18.517252 9685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:18.517535 9688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:18.509320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:59519 (local address 127.0.0.1:37151)
0808 14:43:18.509899 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:18.509916 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:18.509977 (+ 61us) server_negotiation.cc:407] Connection header received
0808 14:43:18.510717 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:18.510739 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:18.511183 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:18.511534 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:18.511979 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:18.512998 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:18.514005 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:18.514499 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:18.514679 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:18.515691 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:18.515696 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:18.515710 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:18.516129 (+ 419us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:18.516187 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:18.516211 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:18.516359 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:18.516558 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 14:43:18.516710 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":184,"threads_started":1}
I20260808 14:43:18.518285 9685 log.cc:824] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:18.520666 9685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2: No bootstrap required, opened a new log
I20260808 14:43:18.520936 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:18.525964 9685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942737a005794d59a7e214d6369d77d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37151 } }
I20260808 14:43:18.526228 9685 raft_consensus.cc:374] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:18.526292 9685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 942737a005794d59a7e214d6369d77d2, State: Initialized, Role: FOLLOWER
I20260808 14:43:18.526841 9685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942737a005794d59a7e214d6369d77d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37151 } }
I20260808 14:43:18.527014 9685 raft_consensus.cc:388] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:18.527092 9685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:18.527228 9685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:18.528246 9685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942737a005794d59a7e214d6369d77d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37151 } }
I20260808 14:43:18.528508 9688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:18.523560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:55993 (local address 127.0.0.1:37151)
0808 14:43:18.523742 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:18.523747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:18.524089 (+ 342us) server_negotiation.cc:407] Connection header received
0808 14:43:18.524227 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:18.524230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:18.524313 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:18.524396 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:18.524799 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:18.525574 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:18.526650 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:18.526933 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:18.527056 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:18.527991 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:18.527994 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:18.527997 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:18.528012 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:18.528048 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:18.528059 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:18.528158 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:18.528291 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 14:43:18.528334 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 14:43:18.528699 9685 leader_election.cc:302] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 942737a005794d59a7e214d6369d77d2; no voters:
I20260808 14:43:18.529099 9685 leader_election.cc:288] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:18.529212 9695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:18.529469 9695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [term 1 LEADER]: Becoming Leader. State: Replica: 942737a005794d59a7e214d6369d77d2, State: Running, Role: LEADER
I20260808 14:43:18.529868 9695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942737a005794d59a7e214d6369d77d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37151 } }
I20260808 14:43:18.530527 9685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:18.531699 9698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 942737a005794d59a7e214d6369d77d2. Latest consensus state: current_term: 1 leader_uuid: "942737a005794d59a7e214d6369d77d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942737a005794d59a7e214d6369d77d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37151 } } }
I20260808 14:43:18.531697 9697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "942737a005794d59a7e214d6369d77d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942737a005794d59a7e214d6369d77d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37151 } } }
I20260808 14:43:18.531864 9697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:18.531864 9698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2 [sys.catalog]: This master's current role is: LEADER
W20260808 14:43:18.535719 9711 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:43:18.535837 9711 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:43:18.535943 9705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:18.536762 9705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:18.543068 9705 catalog_manager.cc:1347] Generated new cluster ID: caffe3f4a5c34899835d98741c7b865b
I20260808 14:43:18.543164 9705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:18.572867 9705 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:18.573997 9705 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:18.586565 9705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 942737a005794d59a7e214d6369d77d2: Generated new TSK 0
I20260808 14:43:18.587391 9705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:18.597990 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 9615
W20260808 14:43:18.605583 9597 connection.cc:535] client connection to 127.0.0.1:37151 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37151): Connection reset by peer (error 104)
2026-08-08T14:43:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1510 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:18 dist-test-slave-p3t5 krb5kdc[9718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:18 dist-test-slave-p3t5 krb5kdc[9718](info): set up 2 sockets
Aug 08 14:43:18 dist-test-slave-p3t5 krb5kdc[9718](info): commencing operation
krb5kdc: starting...
W20260808 14:43:19.699649 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:19 dist-test-slave-p3t5 krb5kdc[9718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 14:43:19.841266 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38963
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:19.962249 9731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:19.962561 9731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:19.962639 9731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:19.966791 9731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:19.966887 9731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:19.966907 9731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:19.966925 9731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:19.966940 9731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:19.972534 9731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38963
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:19.973793 9731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:19.974931 9731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:19.984601 9738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:19.984663 9740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:19.984601 9736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:19.985064 9731 server_base.cc:1034] running on GCE node
I20260808 14:43:19.985808 9731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:19.987074 9731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:19.988282 9731 hybrid_clock.cc:648] HybridClock initialized: now 1786200199988263 us; error 34 us; skew 500 ppm
Aug 08 14:43:19 dist-test-slave-p3t5 krb5kdc[9718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200199, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:19.991456 9731 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 14:43:19.992501 9731 webserver.cc:460] Webserver started at http://127.0.0.1:34609/ using document root <none> and password file <none>
I20260808 14:43:19.993175 9731 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:19.993275 9731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:19.993539 9731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:19.995864 9731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "04703e1948804ede8f58eedf11e7cf66"
format_stamp: "Formatted at 2026-08-08 14:43:19 on dist-test-slave-p3t5"
server_key: "eff5db21da41a9f08e79f430e4759ac3"
server_key_iv: "3956f0751fd1a798e3078773fb9e4437"
server_key_version: "encryptionkey@0"
I20260808 14:43:19.996501 9731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "04703e1948804ede8f58eedf11e7cf66"
format_stamp: "Formatted at 2026-08-08 14:43:19 on dist-test-slave-p3t5"
server_key: "eff5db21da41a9f08e79f430e4759ac3"
server_key_iv: "3956f0751fd1a798e3078773fb9e4437"
server_key_version: "encryptionkey@0"
I20260808 14:43:20.001179 9731 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:43:20.004081 9747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:20.005596 9731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:43:20.005838 9731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "04703e1948804ede8f58eedf11e7cf66"
format_stamp: "Formatted at 2026-08-08 14:43:19 on dist-test-slave-p3t5"
server_key: "eff5db21da41a9f08e79f430e4759ac3"
server_key_iv: "3956f0751fd1a798e3078773fb9e4437"
server_key_version: "encryptionkey@0"
I20260808 14:43:20.005980 9731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:20.011356 9731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:20.015206 9731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:20.015444 9731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:20.023186 9731 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38963
I20260808 14:43:20.023216 9801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38963 every 8 connection(s)
I20260808 14:43:20.024305 9731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:20.027526 9802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:20.029804 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 9731
I20260808 14:43:20.029990 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:20.030316 20033 external_mini_cluster.cc:1428] Setting key c5dff10bf06b83daa453de1ace5fb0e9
I20260808 14:43:20.034847 9802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66: Bootstrap starting.
I20260808 14:43:20.036823 9805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:20.031922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:53921 (local address 127.0.0.1:38963)
0808 14:43:20.032429 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:20.032452 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:20.032501 (+ 49us) server_negotiation.cc:407] Connection header received
0808 14:43:20.033513 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:20.033540 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:20.033981 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:20.034386 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:20.035285 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:20.035290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:20.035307 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:20.035320 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:20.035389 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:20.035416 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:20.035569 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:20.035643 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 14:43:20.035809 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":147,"threads_started":1}
I20260808 14:43:20.037849 9802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:20.038661 9802 log.cc:824] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:20.040058 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:20.041031 9802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66: No bootstrap required, opened a new log
I20260808 14:43:20.044281 9805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:20.042376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:51285 (local address 127.0.0.1:38963)
0808 14:43:20.042513 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:20.042517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:20.042710 (+ 193us) server_negotiation.cc:407] Connection header received
0808 14:43:20.042883 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:20.042886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:20.042929 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:20.043031 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:20.043810 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:20.043814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:20.043816 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:20.043829 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:20.043896 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:20.043907 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:20.044008 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:20.044080 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 14:43:20.044128 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 14:43:20.045846 9802 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04703e1948804ede8f58eedf11e7cf66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38963 } }
I20260808 14:43:20.046103 9802 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:20.046180 9802 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04703e1948804ede8f58eedf11e7cf66, State: Initialized, Role: FOLLOWER
I20260808 14:43:20.046692 9802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04703e1948804ede8f58eedf11e7cf66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38963 } }
I20260808 14:43:20.046881 9802 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:20.046973 9802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:20.047109 9802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:20.048192 9802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04703e1948804ede8f58eedf11e7cf66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38963 } }
I20260808 14:43:20.048619 9802 leader_election.cc:302] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04703e1948804ede8f58eedf11e7cf66; no voters:
I20260808 14:43:20.049036 9802 leader_election.cc:288] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:20.049221 9812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:20.049484 9812 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [term 1 LEADER]: Becoming Leader. State: Replica: 04703e1948804ede8f58eedf11e7cf66, State: Running, Role: LEADER
I20260808 14:43:20.049922 9812 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04703e1948804ede8f58eedf11e7cf66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38963 } }
I20260808 14:43:20.050493 9802 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:20.051746 9813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04703e1948804ede8f58eedf11e7cf66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04703e1948804ede8f58eedf11e7cf66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38963 } } }
I20260808 14:43:20.051898 9813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:20.052228 9814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04703e1948804ede8f58eedf11e7cf66. Latest consensus state: current_term: 1 leader_uuid: "04703e1948804ede8f58eedf11e7cf66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04703e1948804ede8f58eedf11e7cf66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38963 } } }
I20260808 14:43:20.052325 9821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:20.052361 9814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:20.056245 9821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:20.063211 9821 catalog_manager.cc:1347] Generated new cluster ID: c814b4635f044d23816d586f99d15b58
I20260808 14:43:20.063313 9821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:20.106914 9821 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:20.107342 9821 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:20.114249 9821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04703e1948804ede8f58eedf11e7cf66: Generated new TSK 0
I20260808 14:43:20.115073 9821 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:20.128135 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 9731
2026-08-08T14:43:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1529 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:20 dist-test-slave-p3t5 krb5kdc[9833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:20 dist-test-slave-p3t5 krb5kdc[9833](info): set up 2 sockets
Aug 08 14:43:20 dist-test-slave-p3t5 krb5kdc[9833](info): commencing operation
krb5kdc: starting...
W20260808 14:43:21.203509 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:21 dist-test-slave-p3t5 krb5kdc[9833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:43:21.345862 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:37795
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:44881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:37795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:21.465929 9847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:21.466289 9847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:21.466387 9847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:21.470562 9847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:21.470695 9847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:21.470733 9847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:21.470775 9847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:21.470805 9847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:21.476486 9847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:44881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:37795
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:37795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:21.477821 9847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:21.479001 9847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:21.488679 9855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:21.488679 9853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:21.488745 9852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:21.489254 9847 server_base.cc:1034] running on GCE node
I20260808 14:43:21.490038 9847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:21.491321 9847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:21.492545 9847 hybrid_clock.cc:648] HybridClock initialized: now 1786200201492509 us; error 42 us; skew 500 ppm
Aug 08 14:43:21 dist-test-slave-p3t5 krb5kdc[9833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200201, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:21.495719 9847 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:43:21.496738 9847 webserver.cc:460] Webserver started at http://127.19.144.126:35979/ using document root <none> and password file <none>
I20260808 14:43:21.497376 9847 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:21.497481 9847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:21.497798 9847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:21.500023 9847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "53189184c1e64ad4a6d954956a4ee43d"
format_stamp: "Formatted at 2026-08-08 14:43:21 on dist-test-slave-p3t5"
server_key: "957940eda0698cd35451acd2e3aebde1"
server_key_iv: "47e7dc8dc08a09a89ee0c16d6665ae2b"
server_key_version: "encryptionkey@0"
I20260808 14:43:21.500674 9847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "53189184c1e64ad4a6d954956a4ee43d"
format_stamp: "Formatted at 2026-08-08 14:43:21 on dist-test-slave-p3t5"
server_key: "957940eda0698cd35451acd2e3aebde1"
server_key_iv: "47e7dc8dc08a09a89ee0c16d6665ae2b"
server_key_version: "encryptionkey@0"
I20260808 14:43:21.504731 9847 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:43:21.507552 9862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:21.508922 9847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:43:21.509092 9847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "53189184c1e64ad4a6d954956a4ee43d"
format_stamp: "Formatted at 2026-08-08 14:43:21 on dist-test-slave-p3t5"
server_key: "957940eda0698cd35451acd2e3aebde1"
server_key_iv: "47e7dc8dc08a09a89ee0c16d6665ae2b"
server_key_version: "encryptionkey@0"
I20260808 14:43:21.509223 9847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:21.522758 9847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:21.526072 9847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:21.526300 9847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:21.534068 9847 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:37795
I20260808 14:43:21.534078 9915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:37795 every 8 connection(s)
I20260808 14:43:21.535198 9847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:21.539108 9916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:21.544539 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 9847
I20260808 14:43:21.544672 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:21.544986 20033 external_mini_cluster.cc:1428] Setting key bf536ac78a43a6f97e7b86f8c98497cb
I20260808 14:43:21.545544 9916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d: Bootstrap starting.
I20260808 14:43:21.548346 9916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:21.549405 9916 log.cc:824] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:21.551838 9916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d: No bootstrap required, opened a new log
Aug 08 14:43:21 dist-test-slave-p3t5 krb5kdc[9833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:43:21.556491 9916 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53189184c1e64ad4a6d954956a4ee43d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37795 } }
I20260808 14:43:21.556852 9916 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:21.556946 9916 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53189184c1e64ad4a6d954956a4ee43d, State: Initialized, Role: FOLLOWER
I20260808 14:43:21.557542 9916 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53189184c1e64ad4a6d954956a4ee43d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37795 } }
I20260808 14:43:21.557775 9916 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:21.557852 9916 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:21.557994 9916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:21.558974 9916 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53189184c1e64ad4a6d954956a4ee43d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37795 } }
I20260808 14:43:21.559348 9916 leader_election.cc:302] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53189184c1e64ad4a6d954956a4ee43d; no voters:
I20260808 14:43:21.559737 9916 leader_election.cc:288] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:21.559859 9921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:21.560070 9921 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [term 1 LEADER]: Becoming Leader. State: Replica: 53189184c1e64ad4a6d954956a4ee43d, State: Running, Role: LEADER
I20260808 14:43:21.560446 9921 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53189184c1e64ad4a6d954956a4ee43d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37795 } }
I20260808 14:43:21.561182 9916 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:21.561559 9919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:21.546552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37013 (local address 127.19.144.126:37795)
0808 14:43:21.547077 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:21.547094 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:21.547141 (+ 47us) server_negotiation.cc:407] Connection header received
0808 14:43:21.547843 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:21.547861 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:21.548242 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:21.548628 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:21.549607 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:21.550775 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:21.552003 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:21.552513 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:21.552641 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:21.555687 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:21.555693 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:21.555709 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:21.555741 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:21.557978 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:21.558644 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:21.558651 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:21.558658 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:21.558798 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:21.559213 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:21.559217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:21.559218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:21.559729 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:21.559979 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:21.560372 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 14:43:21.560652 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":184,"threads_started":1}
I20260808 14:43:21.562314 9923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53189184c1e64ad4a6d954956a4ee43d. Latest consensus state: current_term: 1 leader_uuid: "53189184c1e64ad4a6d954956a4ee43d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53189184c1e64ad4a6d954956a4ee43d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37795 } } }
I20260808 14:43:21.562450 9923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:21.564129 9922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53189184c1e64ad4a6d954956a4ee43d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53189184c1e64ad4a6d954956a4ee43d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 37795 } } }
I20260808 14:43:21.564246 9922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:21.564353 9930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:21.565501 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:21.568025 9930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:21.574584 9930 catalog_manager.cc:1347] Generated new cluster ID: 7af9b95988ef497a88ec1d7b505b74e6
I20260808 14:43:21.574674 9930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:21.578985 9919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:21.567926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52995 (local address 127.19.144.126:37795)
0808 14:43:21.568043 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:21.568050 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:21.569953 (+ 1903us) server_negotiation.cc:407] Connection header received
0808 14:43:21.570130 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:21.570133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:21.570186 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:21.570270 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:21.570992 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:21.571890 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:21.572927 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:21.573204 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:21.573320 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:21.575256 (+ 1936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:21.575260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:21.575263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:21.575292 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:21.577003 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:21.577552 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:21.577555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:21.577557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:21.577669 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:21.578010 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:21.578013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:21.578015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:21.578215 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:21.578329 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:21.578719 (+ 390us) server_negotiation.cc:299] Negotiation successful
0808 14:43:21.578821 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 14:43:21.595142 9930 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:21.596537 9930 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:21.607622 9930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53189184c1e64ad4a6d954956a4ee43d: Generated new TSK 0
I20260808 14:43:21.608598 9930 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:21.613762 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 9847
W20260808 14:43:21.621822 9831 connection.cc:535] client connection to 127.19.144.126:37795 recv error: Network error: failed to read from TLS socket (remote: 127.19.144.126:37795): Connection reset by peer (error 104)
2026-08-08T14:43:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1486 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:21 dist-test-slave-p3t5 krb5kdc[9948](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:21 dist-test-slave-p3t5 krb5kdc[9948](info): set up 2 sockets
Aug 08 14:43:21 dist-test-slave-p3t5 krb5kdc[9948](info): commencing operation
krb5kdc: starting...
W20260808 14:43:22.679011 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:22 dist-test-slave-p3t5 krb5kdc[9948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:43:22.822448 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:39421
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:35355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:39421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:22.941352 9961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:22.941749 9961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:22.941845 9961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:22.946030 9961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:22.946151 9961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:22.946187 9961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:22.946223 9961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:22.946259 9961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:22.951988 9961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:35355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:39421
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:39421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:22.953207 9961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:22.954389 9961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:22.963809 9966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:22.963832 9969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:22.963873 9967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:22.964215 9961 server_base.cc:1034] running on GCE node
I20260808 14:43:22.965020 9961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:22.966316 9961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:22.967552 9961 hybrid_clock.cc:648] HybridClock initialized: now 1786200202967528 us; error 37 us; skew 500 ppm
Aug 08 14:43:22 dist-test-slave-p3t5 krb5kdc[9948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200202, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:22.970726 9961 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:43:22.971800 9961 webserver.cc:460] Webserver started at http://127.19.144.126:38991/ using document root <none> and password file <none>
I20260808 14:43:22.972437 9961 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:22.972536 9961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:22.972796 9961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:22.975042 9961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "f4f4a0ae01164e959f93846542efd050"
format_stamp: "Formatted at 2026-08-08 14:43:22 on dist-test-slave-p3t5"
server_key: "c6b359a5349eb6d40be8056746f16adc"
server_key_iv: "ec9440c4919386942f0adbbb478e34cc"
server_key_version: "encryptionkey@0"
I20260808 14:43:22.975634 9961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "f4f4a0ae01164e959f93846542efd050"
format_stamp: "Formatted at 2026-08-08 14:43:22 on dist-test-slave-p3t5"
server_key: "c6b359a5349eb6d40be8056746f16adc"
server_key_iv: "ec9440c4919386942f0adbbb478e34cc"
server_key_version: "encryptionkey@0"
I20260808 14:43:22.979704 9961 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 14:43:22.982537 9976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:22.983863 9961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:43:22.984069 9961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "f4f4a0ae01164e959f93846542efd050"
format_stamp: "Formatted at 2026-08-08 14:43:22 on dist-test-slave-p3t5"
server_key: "c6b359a5349eb6d40be8056746f16adc"
server_key_iv: "ec9440c4919386942f0adbbb478e34cc"
server_key_version: "encryptionkey@0"
I20260808 14:43:22.984210 9961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:22.999786 9961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:23.003131 9961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:23.003367 9961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:23.011493 9961 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:39421
I20260808 14:43:23.011510 10029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:39421 every 8 connection(s)
I20260808 14:43:23.012567 9961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:23.015555 10030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:23.021085 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 9961
I20260808 14:43:23.021216 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:23.021492 20033 external_mini_cluster.cc:1428] Setting key ec99738f1eb49cfe21c22f4d6cdb40f6
I20260808 14:43:23.021697 10030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050: Bootstrap starting.
I20260808 14:43:23.024655 10030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:23.025604 10030 log.cc:824] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:23.027915 10030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050: No bootstrap required, opened a new log
I20260808 14:43:23.032374 10033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:23.023068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42059 (local address 127.19.144.126:39421)
0808 14:43:23.023539 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:23.023560 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:23.023604 (+ 44us) server_negotiation.cc:407] Connection header received
0808 14:43:23.024471 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:23.024499 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:23.024998 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:23.025379 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:23.025861 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:23.027284 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:23.028296 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:23.028995 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:23.029166 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:23.030173 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:23.030180 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:23.030198 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:23.030740 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:23.030822 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:23.030860 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:23.031036 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:23.031174 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 14:43:23.031339 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":135,"threads_started":1}
I20260808 14:43:23.032649 10030 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4f4a0ae01164e959f93846542efd050" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39421 } }
I20260808 14:43:23.032930 10030 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:23.033025 10030 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4f4a0ae01164e959f93846542efd050, State: Initialized, Role: FOLLOWER
I20260808 14:43:23.033480 10030 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4f4a0ae01164e959f93846542efd050" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39421 } }
I20260808 14:43:23.033689 10030 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:23.033767 10030 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:23.033869 10030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:23.035109 10030 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4f4a0ae01164e959f93846542efd050" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39421 } }
I20260808 14:43:23.035597 10030 leader_election.cc:302] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4f4a0ae01164e959f93846542efd050; no voters:
I20260808 14:43:23.035995 10030 leader_election.cc:288] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:23.036052 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:23.036139 10035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:23.036363 10035 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [term 1 LEADER]: Becoming Leader. State: Replica: f4f4a0ae01164e959f93846542efd050, State: Running, Role: LEADER
I20260808 14:43:23.036715 10035 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4f4a0ae01164e959f93846542efd050" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39421 } }
I20260808 14:43:23.037350 10030 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:23.038569 10036 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4f4a0ae01164e959f93846542efd050" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4f4a0ae01164e959f93846542efd050" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39421 } } }
I20260808 14:43:23.038717 10036 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:23.038973 10036 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4f4a0ae01164e959f93846542efd050. Latest consensus state: current_term: 1 leader_uuid: "f4f4a0ae01164e959f93846542efd050" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4f4a0ae01164e959f93846542efd050" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 39421 } } }
I20260808 14:43:23.039055 10036 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:23.039455 10049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:23.043876 10049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:23.044041 10033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:23.039153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58129 (local address 127.19.144.126:39421)
0808 14:43:23.039275 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:23.039280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:23.039533 (+ 253us) server_negotiation.cc:407] Connection header received
0808 14:43:23.039749 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:23.039753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:23.039815 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:23.039904 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:23.040221 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:23.040858 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:23.042028 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:23.042373 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:23.042503 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:23.043495 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:23.043499 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:23.043502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:23.043517 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:23.043548 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:23.043561 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:23.043660 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:23.043861 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 14:43:23.043899 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 14:43:23.050156 10049 catalog_manager.cc:1347] Generated new cluster ID: 4c8aec9a77964205bdadc8db9e3b71de
I20260808 14:43:23.050251 10049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:23.075145 10049 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:23.076217 10049 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:23.085573 10049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4f4a0ae01164e959f93846542efd050: Generated new TSK 0
I20260808 14:43:23.086508 10049 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:23.096124 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 9961
W20260808 14:43:23.103720 9946 connection.cc:535] client connection to 127.19.144.126:39421 recv error: Network error: failed to read from TLS socket (remote: 127.19.144.126:39421): Connection reset by peer (error 104)
2026-08-08T14:43:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1481 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:23 dist-test-slave-p3t5 krb5kdc[10061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:23 dist-test-slave-p3t5 krb5kdc[10061](info): set up 2 sockets
Aug 08 14:43:23 dist-test-slave-p3t5 krb5kdc[10061](info): commencing operation
krb5kdc: starting...
W20260808 14:43:24.159909 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:24 dist-test-slave-p3t5 krb5kdc[10061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:43:24.333994 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:46201
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:46201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:24.453145 10074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:24.453506 10074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:24.453591 10074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:24.457726 10074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:24.457841 10074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:24.457881 10074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:24.457921 10074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:24.457957 10074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:24.463428 10074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:46201
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:46201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:24.464646 10074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:24.465783 10074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:24.475173 10080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:24.475153 10079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:24.475196 10074 server_base.cc:1034] running on GCE node
W20260808 14:43:24.475203 10082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:24.476181 10074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:24.477490 10074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:24.478698 10074 hybrid_clock.cc:648] HybridClock initialized: now 1786200204478662 us; error 55 us; skew 500 ppm
Aug 08 14:43:24 dist-test-slave-p3t5 krb5kdc[10061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200204, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:24.481832 10074 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:43:24.482780 10074 webserver.cc:460] Webserver started at http://127.19.144.126:41499/ using document root <none> and password file <none>
I20260808 14:43:24.483359 10074 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:24.483422 10074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:24.483610 10074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:24.485868 10074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "021b413dac3a4cc9aed2708f81fb4bdf"
format_stamp: "Formatted at 2026-08-08 14:43:24 on dist-test-slave-p3t5"
server_key: "d1d81ddadeb2e384708d139026d6f428"
server_key_iv: "e97c6cf9e78ed1d63123a4b3d4b3bc61"
server_key_version: "encryptionkey@0"
I20260808 14:43:24.486443 10074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "021b413dac3a4cc9aed2708f81fb4bdf"
format_stamp: "Formatted at 2026-08-08 14:43:24 on dist-test-slave-p3t5"
server_key: "d1d81ddadeb2e384708d139026d6f428"
server_key_iv: "e97c6cf9e78ed1d63123a4b3d4b3bc61"
server_key_version: "encryptionkey@0"
I20260808 14:43:24.490530 10074 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:43:24.493199 10090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:24.494614 10074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:43:24.494797 10074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "021b413dac3a4cc9aed2708f81fb4bdf"
format_stamp: "Formatted at 2026-08-08 14:43:24 on dist-test-slave-p3t5"
server_key: "d1d81ddadeb2e384708d139026d6f428"
server_key_iv: "e97c6cf9e78ed1d63123a4b3d4b3bc61"
server_key_version: "encryptionkey@0"
I20260808 14:43:24.494931 10074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:24.499934 10074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:24.503479 10074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:24.503698 10074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:24.511001 10074 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:46201
I20260808 14:43:24.511098 10144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:46201 every 8 connection(s)
I20260808 14:43:24.512101 10074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:24.515285 10145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:24.521138 10145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf: Bootstrap starting.
I20260808 14:43:24.522261 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 10074
I20260808 14:43:24.522434 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:24.522733 20033 external_mini_cluster.cc:1428] Setting key fbf237f0f498c9ae5aa739ba0cfcde02
I20260808 14:43:24.524256 10145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:24.525043 10145 log.cc:824] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:24.527539 10145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf: No bootstrap required, opened a new log
I20260808 14:43:24.528354 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:24.523974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54189 (local address 127.19.144.126:46201)
0808 14:43:24.524526 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:24.524542 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:24.524581 (+ 39us) server_negotiation.cc:407] Connection header received
0808 14:43:24.525350 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:24.525371 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:24.525728 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:24.526036 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:24.526970 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:24.526975 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:24.526989 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:24.527000 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:24.527056 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:24.527075 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:24.527167 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:24.527328 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 14:43:24.527511 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":201,"threads_started":1}
I20260808 14:43:24.531535 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:24.531865 10145 raft_consensus.cc:348] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021b413dac3a4cc9aed2708f81fb4bdf" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46201 } }
I20260808 14:43:24.532191 10145 raft_consensus.cc:374] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:24.532253 10145 raft_consensus.cc:729] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 021b413dac3a4cc9aed2708f81fb4bdf, State: Initialized, Role: FOLLOWER
I20260808 14:43:24.532755 10145 consensus_queue.cc:260] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021b413dac3a4cc9aed2708f81fb4bdf" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46201 } }
I20260808 14:43:24.532979 10145 raft_consensus.cc:388] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:24.533073 10145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:24.533245 10145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:24.534489 10145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021b413dac3a4cc9aed2708f81fb4bdf" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46201 } }
I20260808 14:43:24.534930 10145 leader_election.cc:302] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 021b413dac3a4cc9aed2708f81fb4bdf; no voters:
I20260808 14:43:24.535315 10145 leader_election.cc:288] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:24.535439 10155 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:24.535754 10155 raft_consensus.cc:686] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [term 1 LEADER]: Becoming Leader. State: Replica: 021b413dac3a4cc9aed2708f81fb4bdf, State: Running, Role: LEADER
I20260808 14:43:24.536113 10155 consensus_queue.cc:237] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021b413dac3a4cc9aed2708f81fb4bdf" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46201 } }
I20260808 14:43:24.536253 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:24.534230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47765 (local address 127.19.144.126:46201)
0808 14:43:24.534396 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:24.534400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:24.534617 (+ 217us) server_negotiation.cc:407] Connection header received
0808 14:43:24.534745 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:24.534748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:24.534784 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:24.534853 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:24.535736 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:24.535740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:24.535742 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:24.535754 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:24.535781 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:24.535794 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:24.535848 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:24.536013 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 14:43:24.536054 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 14:43:24.536803 10145 sys_catalog.cc:564] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:24.538007 10156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "021b413dac3a4cc9aed2708f81fb4bdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021b413dac3a4cc9aed2708f81fb4bdf" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46201 } } }
I20260808 14:43:24.538141 10156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:24.538570 10164 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:24.542070 10164 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:24.538525 10157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 021b413dac3a4cc9aed2708f81fb4bdf. Latest consensus state: current_term: 1 leader_uuid: "021b413dac3a4cc9aed2708f81fb4bdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021b413dac3a4cc9aed2708f81fb4bdf" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46201 } } }
I20260808 14:43:24.542805 10157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:24.548257 10164 catalog_manager.cc:1347] Generated new cluster ID: 1d8542a0d83a4d57806fd5d9cabc589b
I20260808 14:43:24.548331 10164 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:24.575872 10164 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:24.576251 10164 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:24.585144 10164 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 021b413dac3a4cc9aed2708f81fb4bdf: Generated new TSK 0
I20260808 14:43:24.585947 10164 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:24.592507 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 10074
2026-08-08T14:43:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1496 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:24 dist-test-slave-p3t5 krb5kdc[10176](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:24 dist-test-slave-p3t5 krb5kdc[10176](info): set up 2 sockets
Aug 08 14:43:24 dist-test-slave-p3t5 krb5kdc[10176](info): commencing operation
krb5kdc: starting...
W20260808 14:43:25.659581 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:25 dist-test-slave-p3t5 krb5kdc[10176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:43:25.806847 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:36743
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:34695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:36743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:25.928071 10189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:25.928453 10189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:25.928532 10189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:25.932802 10189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:25.932932 10189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:25.932971 10189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:25.933012 10189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:25.933043 10189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:25.938784 10189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:34695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:36743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:36743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:25.940050 10189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:25.941252 10189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:25.951102 10194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:25.951113 10197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:25.951113 10195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:25.951444 10189 server_base.cc:1034] running on GCE node
I20260808 14:43:25.952098 10189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:25.953372 10189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:25.954569 10189 hybrid_clock.cc:648] HybridClock initialized: now 1786200205954557 us; error 66 us; skew 500 ppm
Aug 08 14:43:25 dist-test-slave-p3t5 krb5kdc[10176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200205, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:25.957587 10189 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:43:25.958619 10189 webserver.cc:460] Webserver started at http://127.19.144.126:45697/ using document root <none> and password file <none>
I20260808 14:43:25.959187 10189 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:25.959249 10189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:25.959436 10189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:25.961688 10189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "7bb751c17700422088a80b011e7b6879"
format_stamp: "Formatted at 2026-08-08 14:43:25 on dist-test-slave-p3t5"
server_key: "b623aebe42eb49dec8837581eb9f95b9"
server_key_iv: "60593b5319a95861946f916af8e15ef5"
server_key_version: "encryptionkey@0"
I20260808 14:43:25.962307 10189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "7bb751c17700422088a80b011e7b6879"
format_stamp: "Formatted at 2026-08-08 14:43:25 on dist-test-slave-p3t5"
server_key: "b623aebe42eb49dec8837581eb9f95b9"
server_key_iv: "60593b5319a95861946f916af8e15ef5"
server_key_version: "encryptionkey@0"
I20260808 14:43:25.966475 10189 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 14:43:25.969313 10204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:25.970795 10189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:43:25.970971 10189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "7bb751c17700422088a80b011e7b6879"
format_stamp: "Formatted at 2026-08-08 14:43:25 on dist-test-slave-p3t5"
server_key: "b623aebe42eb49dec8837581eb9f95b9"
server_key_iv: "60593b5319a95861946f916af8e15ef5"
server_key_version: "encryptionkey@0"
I20260808 14:43:25.971110 10189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:26.000969 10189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:26.005198 10189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:26.005470 10189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:26.014254 10189 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:36743
I20260808 14:43:26.014286 10258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:36743 every 8 connection(s)
I20260808 14:43:26.015497 10189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:26.016178 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 10189
I20260808 14:43:26.016373 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:26.016718 20033 external_mini_cluster.cc:1428] Setting key 9c09849468c163f4e2a95fabc1b5bf93
I20260808 14:43:26.019099 10259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:26.025763 10259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879: Bootstrap starting.
Aug 08 14:43:26 dist-test-slave-p3t5 krb5kdc[10176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:43:26.028913 10259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:26.029887 10259 log.cc:824] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:26.032259 10259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879: No bootstrap required, opened a new log
I20260808 14:43:26.032635 10262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:26.018224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52295 (local address 127.19.144.126:36743)
0808 14:43:26.018673 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:26.018695 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:26.018930 (+ 235us) server_negotiation.cc:407] Connection header received
0808 14:43:26.019777 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:26.019796 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:26.020153 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:26.020504 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:26.021342 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:26.022454 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:26.023368 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:26.023880 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:26.024058 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:26.027358 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:26.027365 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:26.027380 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:26.027420 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:26.029427 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:26.030006 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:26.030012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:26.030043 (+ 31us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:26.030184 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:26.030501 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:26.030505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:26.030506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:26.030954 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:26.031168 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:26.031495 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 14:43:26.031824 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":147,"threads_started":1}
I20260808 14:43:26.035737 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:26.036774 10259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb751c17700422088a80b011e7b6879" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36743 } }
I20260808 14:43:26.037137 10259 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:26.037214 10259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bb751c17700422088a80b011e7b6879, State: Initialized, Role: FOLLOWER
I20260808 14:43:26.037868 10259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb751c17700422088a80b011e7b6879" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36743 } }
I20260808 14:43:26.038086 10259 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:26.038174 10259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:26.038296 10259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:26.039441 10259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb751c17700422088a80b011e7b6879" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36743 } }
I20260808 14:43:26.039865 10259 leader_election.cc:302] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bb751c17700422088a80b011e7b6879; no voters:
I20260808 14:43:26.040279 10259 leader_election.cc:288] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:26.040378 10269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:26.040609 10269 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [term 1 LEADER]: Becoming Leader. State: Replica: 7bb751c17700422088a80b011e7b6879, State: Running, Role: LEADER
I20260808 14:43:26.041045 10269 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb751c17700422088a80b011e7b6879" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36743 } }
I20260808 14:43:26.041921 10259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:26.043701 10271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7bb751c17700422088a80b011e7b6879" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb751c17700422088a80b011e7b6879" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36743 } } }
I20260808 14:43:26.043860 10271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:26.044296 10270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bb751c17700422088a80b011e7b6879. Latest consensus state: current_term: 1 leader_uuid: "7bb751c17700422088a80b011e7b6879" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb751c17700422088a80b011e7b6879" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 36743 } } }
I20260808 14:43:26.044447 10270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:26.047302 10262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:26.037994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37717 (local address 127.19.144.126:36743)
0808 14:43:26.038265 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:26.038273 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:26.038310 (+ 37us) server_negotiation.cc:407] Connection header received
0808 14:43:26.038494 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:26.038497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:26.038539 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:26.038623 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:26.039333 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:26.039923 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:26.041053 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:26.041375 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:26.041438 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:26.043468 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:26.043472 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:26.043475 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:26.043504 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:26.045329 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:26.045993 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:26.045997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:26.045999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:26.046094 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:26.046400 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:26.046403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:26.046405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:26.046598 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:26.046759 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:26.047002 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 14:43:26.047122 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186}
W20260808 14:43:26.048615 10285 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:43:26.048703 10285 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:43:26.048789 10279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:26.049785 10279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:26.055974 10279 catalog_manager.cc:1347] Generated new cluster ID: 87a7bd8267504e1596e3746cdc4acca9
I20260808 14:43:26.056061 10279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:26.075241 10279 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:26.076475 10279 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:26.083252 10279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7bb751c17700422088a80b011e7b6879: Generated new TSK 0
I20260808 14:43:26.084051 10279 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:26.096611 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 10189
2026-08-08T14:43:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1504 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:26 dist-test-slave-p3t5 krb5kdc[10291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:26 dist-test-slave-p3t5 krb5kdc[10291](info): set up 2 sockets
Aug 08 14:43:26 dist-test-slave-p3t5 krb5kdc[10291](info): commencing operation
krb5kdc: starting...
W20260808 14:43:27.167744 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:27 dist-test-slave-p3t5 krb5kdc[10291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:43:27.309216 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:46541
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:45013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:46541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:27.429679 10304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:27.429988 10304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:27.430033 10304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:27.434124 10304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:27.434195 10304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:27.434216 10304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:27.434232 10304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:27.434247 10304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:27.439685 10304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:45013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:46541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:46541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:27.440816 10304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:27.441885 10304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:43:27.451733 10304 server_base.cc:1034] running on GCE node
W20260808 14:43:27.451689 10309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:27.451689 10310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:27.451691 10312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:27.452665 10304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:27.454059 10304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:27.455236 10304 hybrid_clock.cc:648] HybridClock initialized: now 1786200207455224 us; error 42 us; skew 500 ppm
Aug 08 14:43:27 dist-test-slave-p3t5 krb5kdc[10291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200207, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:27.458374 10304 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:43:27.459474 10304 webserver.cc:460] Webserver started at http://127.19.144.126:34737/ using document root <none> and password file <none>
I20260808 14:43:27.460099 10304 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:27.460201 10304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:27.460466 10304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:27.462778 10304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "4d6a228d3dfd445dbc567885cefa8084"
format_stamp: "Formatted at 2026-08-08 14:43:27 on dist-test-slave-p3t5"
server_key: "9ced07793dfc584ab40206f8a62e28c2"
server_key_iv: "8191c2378add7b61f477e9552e497d90"
server_key_version: "encryptionkey@0"
I20260808 14:43:27.463400 10304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "4d6a228d3dfd445dbc567885cefa8084"
format_stamp: "Formatted at 2026-08-08 14:43:27 on dist-test-slave-p3t5"
server_key: "9ced07793dfc584ab40206f8a62e28c2"
server_key_iv: "8191c2378add7b61f477e9552e497d90"
server_key_version: "encryptionkey@0"
I20260808 14:43:27.467653 10304 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:43:27.470388 10319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:27.471678 10304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 14:43:27.471851 10304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "4d6a228d3dfd445dbc567885cefa8084"
format_stamp: "Formatted at 2026-08-08 14:43:27 on dist-test-slave-p3t5"
server_key: "9ced07793dfc584ab40206f8a62e28c2"
server_key_iv: "8191c2378add7b61f477e9552e497d90"
server_key_version: "encryptionkey@0"
I20260808 14:43:27.471980 10304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:27.497692 10304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:27.501003 10304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:27.501235 10304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:27.509099 10304 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:46541
I20260808 14:43:27.509126 10372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:46541 every 8 connection(s)
I20260808 14:43:27.510219 10304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:27.513211 10373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:27.517908 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 10304
I20260808 14:43:27.518076 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:27.518394 20033 external_mini_cluster.cc:1428] Setting key b6c72d5317d672609e282cd28c0402e8
I20260808 14:43:27.520031 10373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084: Bootstrap starting.
I20260808 14:43:27.522683 10373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:27.523514 10373 log.cc:824] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:27.525723 10373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084: No bootstrap required, opened a new log
I20260808 14:43:27.528524 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:27.520088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32843 (local address 127.19.144.126:46541)
0808 14:43:27.520619 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:27.520645 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:27.520684 (+ 39us) server_negotiation.cc:407] Connection header received
0808 14:43:27.521399 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:27.521417 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:27.521981 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:27.522402 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:27.522907 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:27.524056 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:27.525021 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:27.525527 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:27.525749 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:27.526679 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:27.526684 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:27.526699 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:27.527144 (+ 445us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:27.527205 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:27.527239 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:27.527362 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:27.527526 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 14:43:27.527749 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":164,"threads_started":1}
I20260808 14:43:27.529874 10373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d6a228d3dfd445dbc567885cefa8084" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46541 } }
I20260808 14:43:27.530144 10373 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:27.530237 10373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d6a228d3dfd445dbc567885cefa8084, State: Initialized, Role: FOLLOWER
I20260808 14:43:27.530774 10373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d6a228d3dfd445dbc567885cefa8084" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46541 } }
I20260808 14:43:27.530941 10373 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:27.531016 10373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:27.531148 10373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:27.531836 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:27.532234 10373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d6a228d3dfd445dbc567885cefa8084" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46541 } }
I20260808 14:43:27.532598 10373 leader_election.cc:302] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d6a228d3dfd445dbc567885cefa8084; no voters:
I20260808 14:43:27.532990 10373 leader_election.cc:288] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:27.533039 10378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:27.533224 10378 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [term 1 LEADER]: Becoming Leader. State: Replica: 4d6a228d3dfd445dbc567885cefa8084, State: Running, Role: LEADER
I20260808 14:43:27.533650 10378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d6a228d3dfd445dbc567885cefa8084" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46541 } }
I20260808 14:43:27.534404 10373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:27.535630 10382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d6a228d3dfd445dbc567885cefa8084" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d6a228d3dfd445dbc567885cefa8084" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46541 } } }
I20260808 14:43:27.535792 10382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:27.536151 10381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d6a228d3dfd445dbc567885cefa8084. Latest consensus state: current_term: 1 leader_uuid: "4d6a228d3dfd445dbc567885cefa8084" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d6a228d3dfd445dbc567885cefa8084" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 46541 } } }
I20260808 14:43:27.536239 10381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:27.536252 10392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:27.539654 10392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:27.540946 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:27.534258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36099 (local address 127.19.144.126:46541)
0808 14:43:27.534391 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:27.534396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:27.534678 (+ 282us) server_negotiation.cc:407] Connection header received
0808 14:43:27.534917 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:27.534921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:27.534970 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:27.535022 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:27.537275 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:27.537965 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:27.539165 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:27.539416 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:27.539539 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:27.540466 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:27.540471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:27.540473 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:27.540488 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:27.540525 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:27.540538 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:27.540624 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:27.540715 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 14:43:27.540771 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 14:43:27.546456 10392 catalog_manager.cc:1347] Generated new cluster ID: 3a24ea497efe44259317ec422a2bb885
I20260808 14:43:27.546547 10392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:27.560281 10392 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:27.561764 10392 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:27.572623 10392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d6a228d3dfd445dbc567885cefa8084: Generated new TSK 0
I20260808 14:43:27.573552 10392 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:27.581493 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 10304
2026-08-08T14:43:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1484 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:27 dist-test-slave-p3t5 krb5kdc[10405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:27 dist-test-slave-p3t5 krb5kdc[10405](info): set up 2 sockets
Aug 08 14:43:27 dist-test-slave-p3t5 krb5kdc[10405](info): commencing operation
krb5kdc: starting...
W20260808 14:43:28.712175 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:28 dist-test-slave-p3t5 krb5kdc[10405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:43:28.862406 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:33233
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:35083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:33233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:28.985606 10418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:28.986037 10418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:28.986083 10418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:28.990341 10418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:28.990422 10418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:28.990442 10418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:28.990459 10418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:28.990475 10418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:28.996167 10418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:35083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:33233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:33233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:28.997401 10418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:28.998598 10418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:43:29.008592 10418 server_base.cc:1034] running on GCE node
W20260808 14:43:29.008528 10423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:29.008507 10427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:29.008538 10425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:29.009552 10418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:29.010951 10418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:29.012190 10418 hybrid_clock.cc:648] HybridClock initialized: now 1786200209012145 us; error 61 us; skew 500 ppm
Aug 08 14:43:29 dist-test-slave-p3t5 krb5kdc[10405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200209, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:29.015614 10418 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:43:29.016786 10418 webserver.cc:460] Webserver started at http://127.19.144.126:34591/ using document root <none> and password file <none>
I20260808 14:43:29.017477 10418 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:29.017586 10418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:29.017910 10418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:29.020306 10418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "57978820236f44c39714e61dea4f073d"
format_stamp: "Formatted at 2026-08-08 14:43:29 on dist-test-slave-p3t5"
server_key: "54a32ebf48febbeed1db394e98a2b425"
server_key_iv: "50966c4c81dff788aab49d775ebc7699"
server_key_version: "encryptionkey@0"
I20260808 14:43:29.020999 10418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "57978820236f44c39714e61dea4f073d"
format_stamp: "Formatted at 2026-08-08 14:43:29 on dist-test-slave-p3t5"
server_key: "54a32ebf48febbeed1db394e98a2b425"
server_key_iv: "50966c4c81dff788aab49d775ebc7699"
server_key_version: "encryptionkey@0"
I20260808 14:43:29.025923 10418 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 14:43:29.028916 10434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:29.030560 10418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:43:29.030702 10418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "57978820236f44c39714e61dea4f073d"
format_stamp: "Formatted at 2026-08-08 14:43:29 on dist-test-slave-p3t5"
server_key: "54a32ebf48febbeed1db394e98a2b425"
server_key_iv: "50966c4c81dff788aab49d775ebc7699"
server_key_version: "encryptionkey@0"
I20260808 14:43:29.030817 10418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:29.035825 10418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:29.039505 10418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:29.039703 10418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:29.047871 10418 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:33233
I20260808 14:43:29.047888 10489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:33233 every 8 connection(s)
I20260808 14:43:29.048964 10418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:29.051189 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 10418
I20260808 14:43:29.051373 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:29.051687 20033 external_mini_cluster.cc:1428] Setting key 7e89049562d491c4fbf11364b2889e0f
I20260808 14:43:29.052677 10490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:29.058282 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:29.053384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37273 (local address 127.19.144.126:33233)
0808 14:43:29.053924 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:29.053945 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:29.053992 (+ 47us) server_negotiation.cc:407] Connection header received
0808 14:43:29.054903 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:29.054932 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:29.055342 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:29.055761 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:29.056689 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:29.056695 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:29.056714 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:29.056728 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:29.056803 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:29.056833 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:29.056986 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:29.057057 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 14:43:29.057215 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":212,"threads_started":1}
I20260808 14:43:29.060483 10490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d: Bootstrap starting.
I20260808 14:43:29.062041 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:29.063683 10490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:29.064877 10490 log.cc:824] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:29.067114 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:29.064640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34289 (local address 127.19.144.126:33233)
0808 14:43:29.064780 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:29.064785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:29.065081 (+ 296us) server_negotiation.cc:407] Connection header received
0808 14:43:29.065400 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:29.065403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:29.065443 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:29.065531 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:29.066580 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:29.066584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:29.066587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 14:43:29.066599 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:29.066631 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 14:43:29.066645 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:29.066759 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:29.066942 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 14:43:29.066986 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:43:29.067919 10490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d: No bootstrap required, opened a new log
I20260808 14:43:29.072055 10490 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57978820236f44c39714e61dea4f073d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33233 } }
I20260808 14:43:29.072310 10490 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 14:43:29.072364 10490 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57978820236f44c39714e61dea4f073d, State: Initialized, Role: FOLLOWER
I20260808 14:43:29.072763 10490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57978820236f44c39714e61dea4f073d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33233 } }
I20260808 14:43:29.072881 10490 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 14:43:29.072923 10490 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 14:43:29.072992 10490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:29.074123 10490 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57978820236f44c39714e61dea4f073d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33233 } }
I20260808 14:43:29.074529 10490 leader_election.cc:302] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57978820236f44c39714e61dea4f073d; no voters:
I20260808 14:43:29.074911 10490 leader_election.cc:288] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 14:43:29.075126 10501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:29.075453 10501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [term 1 LEADER]: Becoming Leader. State: Replica: 57978820236f44c39714e61dea4f073d, State: Running, Role: LEADER
I20260808 14:43:29.075837 10501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57978820236f44c39714e61dea4f073d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33233 } }
I20260808 14:43:29.076265 10490 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:29.077767 10503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57978820236f44c39714e61dea4f073d. Latest consensus state: current_term: 1 leader_uuid: "57978820236f44c39714e61dea4f073d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57978820236f44c39714e61dea4f073d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33233 } } }
I20260808 14:43:29.077806 10502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57978820236f44c39714e61dea4f073d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57978820236f44c39714e61dea4f073d" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33233 } } }
I20260808 14:43:29.077905 10503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:29.077924 10502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:29.078338 10512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:29.081463 10512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:29.087102 10512 catalog_manager.cc:1347] Generated new cluster ID: e9195758ea994994bb342691a9d595c4
I20260808 14:43:29.087193 10512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:29.097884 10512 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:29.098313 10512 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:29.116021 10512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57978820236f44c39714e61dea4f073d: Generated new TSK 0
I20260808 14:43:29.117561 10512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:29.135360 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 10418
2026-08-08T14:43:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1556 ms)
[----------] 18 tests from AuthTokenIssuingTest (26867 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:29 dist-test-slave-p3t5 krb5kdc[10523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:29 dist-test-slave-p3t5 krb5kdc[10523](info): set up 2 sockets
Aug 08 14:43:29 dist-test-slave-p3t5 krb5kdc[10523](info): commencing operation
krb5kdc: starting...
W20260808 14:43:30.243844 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:30 dist-test-slave-p3t5 krb5kdc[10523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:43:30.385694 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:33097
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:33097,127.19.144.125:41803,127.19.144.124:43541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:30.507184 10537 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 14:43:30.507588 10537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:30.507666 10537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:30.507750 10537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:30.512025 10537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:30.512126 10537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:30.512162 10537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:30.512203 10537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:30.512243 10537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:30.517956 10537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:33097,127.19.144.125:41803,127.19.144.124:43541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:33097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:30.519203 10537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:30.520344 10537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:30.530403 10543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:30.530563 10537 server_base.cc:1034] running on GCE node
W20260808 14:43:30.530413 10542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:30.530417 10545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:30.531396 10537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:30.532658 10537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:30.533828 10537 hybrid_clock.cc:648] HybridClock initialized: now 1786200210533793 us; error 53 us; skew 500 ppm
Aug 08 14:43:30 dist-test-slave-p3t5 krb5kdc[10523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200210, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:30.542862 10537 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:43:30.544075 10537 webserver.cc:460] Webserver started at http://127.19.144.126:43889/ using document root <none> and password file <none>
I20260808 14:43:30.544744 10537 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:30.544816 10537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:30.545070 10537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:30.547459 10537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "a7f7f51ae73f445584e6830410982c83"
format_stamp: "Formatted at 2026-08-08 14:43:30 on dist-test-slave-p3t5"
server_key: "ad2db53302c7b78cba9dd55bbb610c57"
server_key_iv: "0343c8b80cdc57f0fe372a70011402db"
server_key_version: "encryptionkey@0"
I20260808 14:43:30.548094 10537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "a7f7f51ae73f445584e6830410982c83"
format_stamp: "Formatted at 2026-08-08 14:43:30 on dist-test-slave-p3t5"
server_key: "ad2db53302c7b78cba9dd55bbb610c57"
server_key_iv: "0343c8b80cdc57f0fe372a70011402db"
server_key_version: "encryptionkey@0"
I20260808 14:43:30.552202 10537 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 14:43:30.555168 10552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:30.556524 10537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 14:43:30.556679 10537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "a7f7f51ae73f445584e6830410982c83"
format_stamp: "Formatted at 2026-08-08 14:43:30 on dist-test-slave-p3t5"
server_key: "ad2db53302c7b78cba9dd55bbb610c57"
server_key_iv: "0343c8b80cdc57f0fe372a70011402db"
server_key_version: "encryptionkey@0"
I20260808 14:43:30.556818 10537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:30.584391 10537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:30.587955 10537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:30.588131 10537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:30.595811 10537 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:33097
I20260808 14:43:30.596128 10606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:33097 every 8 connection(s)
I20260808 14:43:30.596884 10537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:30.599905 10607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:30.604526 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 10537
I20260808 14:43:30.604688 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:30.604364 10607 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:43:30.605043 20033 external_mini_cluster.cc:1428] Setting key 87079f1928ed9da690b7ff71914b267d
WARNING: no policy specified for kudu/127.19.144.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.125@KRBTEST.COM" created.
Aug 08 14:43:30 dist-test-slave-p3t5 krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200210, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:43:30.625211 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.607267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47064 (local address 127.19.144.126:33097)
0808 14:43:30.607900 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:30.607919 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:30.608133 (+ 214us) server_negotiation.cc:407] Connection header received
0808 14:43:30.609474 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:30.609483 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:30.609782 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:30.610009 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:30.610891 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:30.611776 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:30.612861 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:30.613258 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:30.613450 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:30.619005 (+ 5555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:30.619012 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:30.619026 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:30.619051 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:30.621571 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:30.622206 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:30.622215 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:30.622222 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:30.622353 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:30.622685 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:30.622690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:30.622692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:30.623159 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:30.623318 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:30.623710 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 14:43:30.624012 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":489,"spinlock_wait_cycles":58240,"thread_start_us":169,"threads_started":1}
I20260808 14:43:30.625211 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.607267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:33097 (local address 127.0.0.1:47064)
0808 14:43:30.607923 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.607992 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:30.609116 (+ 1124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:30.610025 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:30.610030 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:30.610400 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:30.610719 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:30.610735 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:30.611942 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:30.611944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:30.612696 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:30.612705 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:30.613438 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:30.613443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:30.613538 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:30.614263 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:30.614281 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:30.618802 (+ 4521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:30.621771 (+ 2969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:30.621782 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:30.621796 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:30.622063 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:30.622457 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:30.622462 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:30.622464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:30.622588 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:30.623283 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:30.623298 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:30.623453 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 14:43:30.623644 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 14:43:30.624072 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":113,"thread_start_us":163,"threads_started":1}
I20260808 14:43:30.628528 10607 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:43:30.629477 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.629107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:41803 (local address 127.0.0.1:53526)
0808 14:43:30.629310 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.629397 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:41803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260808 14:43:30.630002 10553 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.144.125:41803: connect: Connection refused (error 111)
I20260808 14:43:30.631171 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.630863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:41803 (local address 127.0.0.1:53530)
0808 14:43:30.631039 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.631111 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:41803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260808 14:43:30.631321 10607 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.125:41803: Network error: Client connection negotiation failed: client connection to 127.19.144.125:41803: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.125.keytab.
Entry for principal kudu/127.19.144.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.125.keytab.
I20260808 14:43:30.638415 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.125:41803
--webserver_interface=127.19.144.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.125.keytab
--principal=kudu/127.19.144.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:33097,127.19.144.125:41803,127.19.144.124:43541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:43:30.680614 10607 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } attempt: 1
I20260808 14:43:30.681680 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.681292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:41803 (local address 127.0.0.1:53542)
0808 14:43:30.681469 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.681544 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:41803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 14:43:30.682463 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.682223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:41803 (local address 127.0.0.1:53558)
0808 14:43:30.682339 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.682392 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:41803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 14:43:30.682593 10607 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.125:41803: Network error: Client connection negotiation failed: client connection to 127.19.144.125:41803: connect: Connection refused (error 111)
I20260808 14:43:30.750793 10607 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } attempt: 2
I20260808 14:43:30.751885 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.751470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:41803 (local address 127.0.0.1:53570)
0808 14:43:30.751678 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.751762 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:41803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 14:43:30.752791 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.752522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:41803 (local address 127.0.0.1:53578)
0808 14:43:30.752654 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.752716 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:41803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 14:43:30.753042 10607 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.125:41803: Network error: Client connection negotiation failed: client connection to 127.19.144.125:41803: connect: Connection refused (error 111)
W20260808 14:43:30.761147 10614 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 14:43:30.761488 10614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:30.761538 10614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:30.761657 10614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:30.766168 10614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:30.766238 10614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:30.766258 10614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:30.766275 10614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:30.766319 10614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:30.772019 10614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:33097,127.19.144.125:41803,127.19.144.124:43541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.125.keytab
--principal=kudu/127.19.144.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.125:41803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.144.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:30.773262 10614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:30.774412 10614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:30.783483 10622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:30.783500 10620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:30.783486 10619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:30.783924 10614 server_base.cc:1034] running on GCE node
I20260808 14:43:30.784636 10614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:30.785979 10614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:30.787158 10614 hybrid_clock.cc:648] HybridClock initialized: now 1786200210787135 us; error 37 us; skew 500 ppm
Aug 08 14:43:30 dist-test-slave-p3t5 krb5kdc[10523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200210, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:30.790251 10614 init.cc:377] Logged in from keytab as kudu/127.19.144.125@KRBTEST.COM (short username kudu)
I20260808 14:43:30.791239 10614 webserver.cc:460] Webserver started at http://127.19.144.125:41245/ using document root <none> and password file <none>
I20260808 14:43:30.791842 10614 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:30.791908 10614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:30.792140 10614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:30.794391 10614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/data/instance:
uuid: "71ba32eefe1b4b10a579a0d9298ec3a2"
format_stamp: "Formatted at 2026-08-08 14:43:30 on dist-test-slave-p3t5"
server_key: "197a2ba467c2f915cbf0bbe90f4a119e"
server_key_iv: "1bad8f1f11968a08c2b0b9841f37ce54"
server_key_version: "encryptionkey@0"
I20260808 14:43:30.795006 10614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/wal/instance:
uuid: "71ba32eefe1b4b10a579a0d9298ec3a2"
format_stamp: "Formatted at 2026-08-08 14:43:30 on dist-test-slave-p3t5"
server_key: "197a2ba467c2f915cbf0bbe90f4a119e"
server_key_iv: "1bad8f1f11968a08c2b0b9841f37ce54"
server_key_version: "encryptionkey@0"
I20260808 14:43:30.799082 10614 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:43:30.802170 10630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:30.803563 10614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:43:30.803728 10614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/wal
uuid: "71ba32eefe1b4b10a579a0d9298ec3a2"
format_stamp: "Formatted at 2026-08-08 14:43:30 on dist-test-slave-p3t5"
server_key: "197a2ba467c2f915cbf0bbe90f4a119e"
server_key_iv: "1bad8f1f11968a08c2b0b9841f37ce54"
server_key_version: "encryptionkey@0"
I20260808 14:43:30.803853 10614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:30.818739 10614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:30.822382 10614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:30.822562 10614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:30.829738 10614 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.125:41803
I20260808 14:43:30.830042 10684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.125:41803 every 8 connection(s)
I20260808 14:43:30.830849 10614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/data/info.pb
I20260808 14:43:30.833948 10685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:30.836920 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 10614
I20260808 14:43:30.837036 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-1/wal/instance
I20260808 14:43:30.837347 20033 external_mini_cluster.cc:1428] Setting key 3350018e4de8d33fe1da91c325603bb4
I20260808 14:43:30.838003 10685 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:43:30.844276 10607 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } attempt: 3
WARNING: no policy specified for kudu/127.19.144.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.124@KRBTEST.COM" created.
Aug 08 14:43:30 dist-test-slave-p3t5 krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200210, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.125@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
Aug 08 14:43:30 dist-test-slave-p3t5 krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200210, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for kudu/127.19.144.125@KRBTEST.COM
I20260808 14:43:30.858527 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.844786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:41803 (local address 127.0.0.1:53586)
0808 14:43:30.844989 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.845007 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:30.845092 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:30.845829 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:30.845835 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:30.846067 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:30.846211 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:30.846220 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:30.847381 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:30.847386 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:30.848275 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:30.848290 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:30.849124 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:30.849128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:30.849247 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:30.849746 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:30.849759 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:30.853850 (+ 4091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:30.856193 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:30.856197 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:30.856199 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:30.856413 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:30.856780 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:30.856784 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:30.856786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:30.856882 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:30.857712 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:30.857719 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:30.857876 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 14:43:30.858012 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 14:43:30.858204 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123}
I20260808 14:43:30.859427 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.840535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47066 (local address 127.19.144.126:33097)
0808 14:43:30.840690 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:30.840694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:30.841859 (+ 1165us) server_negotiation.cc:407] Connection header received
0808 14:43:30.842848 (+ 989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:30.842855 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:30.842931 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:30.843034 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:30.844243 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:30.845152 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:30.846656 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:30.847090 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:30.847217 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:30.854078 (+ 6861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:30.854086 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:30.854093 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:30.854127 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:30.857215 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:30.857799 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:30.857802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:30.857803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:30.857875 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:30.858180 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:30.858183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:30.858185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:30.858362 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:30.858472 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:30.859157 (+ 685us) server_negotiation.cc:299] Negotiation successful
0808 14:43:30.859297 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 14:43:30.860553 10689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.844874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53586 (local address 127.19.144.125:41803)
0808 14:43:30.845321 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:30.845334 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:30.845356 (+ 22us) server_negotiation.cc:407] Connection header received
0808 14:43:30.845396 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:30.845401 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:30.845641 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:30.845810 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:30.846354 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:30.847238 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:30.848415 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:30.849015 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:30.849120 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:30.854057 (+ 4937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:30.854068 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:30.854086 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:30.854110 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:30.856059 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:30.856539 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:30.856552 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:30.856558 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:30.856675 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:30.856983 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:30.856986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:30.856988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:30.857536 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:30.857706 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:30.858947 (+ 1241us) server_negotiation.cc:299] Negotiation successful
0808 14:43:30.859453 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":348,"thread_start_us":146,"threads_started":1}
I20260808 14:43:30.860550 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.841072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:33097 (local address 127.0.0.1:47066)
0808 14:43:30.841696 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.841742 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:30.842564 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:30.843196 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:30.843203 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:30.843568 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:30.844075 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:30.844089 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:30.845295 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:30.845298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:30.846526 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:30.846536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:30.847265 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:30.847272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:30.847490 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:30.848116 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:30.848133 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:30.853914 (+ 5781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:30.857337 (+ 3423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:30.857344 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:30.857350 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:30.857666 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:30.857987 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:30.857990 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:30.857993 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:30.858082 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:30.858478 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:30.858487 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:30.858864 (+ 377us) client_negotiation.cc:769] Sending connection context
0808 14:43:30.859068 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 14:43:30.859548 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":173,"spinlock_wait_cycles":4992,"thread_start_us":157,"threads_started":1}
I20260808 14:43:30.862731 10685 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:43:30.863544 10607 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:43:30.864297 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.864006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:43541 (local address 127.0.0.1:34370)
0808 14:43:30.864128 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.864185 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 14:43:30.865118 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.864791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:43541 (local address 127.0.0.1:34380)
0808 14:43:30.865007 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.865059 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260808 14:43:30.865454 10607 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:43541: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
Aug 08 14:43:30 dist-test-slave-p3t5 krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200210, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.125@KRBTEST.COM for kudu/127.19.144.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.124.keytab.
Entry for principal kudu/127.19.144.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.124.keytab.
I20260808 14:43:30.873070 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.863334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:41803 (local address 127.0.0.1:53598)
0808 14:43:30.863772 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.863792 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:30.863913 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:30.864292 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:30.864295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:30.864494 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:30.864649 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:30.864660 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:30.865586 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:30.865589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:30.866287 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:30.866298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:30.866891 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:30.866894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:30.866992 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:30.867530 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:30.867547 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:30.869000 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:30.871110 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:30.871113 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:30.871115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:30.871339 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:30.871726 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:30.871729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:30.871730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:30.871821 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:30.872317 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:30.872320 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:30.872437 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 14:43:30.872567 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 14:43:30.872754 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249}
I20260808 14:43:30.873554 10689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.863534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53598 (local address 127.19.144.125:41803)
0808 14:43:30.863673 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:30.863678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:30.864009 (+ 331us) server_negotiation.cc:407] Connection header received
0808 14:43:30.864047 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:30.864052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:30.864100 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:30.864168 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:30.864787 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:30.865470 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:30.866412 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:30.866771 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:30.866892 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:30.869152 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:30.869156 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:30.869158 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:30.869181 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:30.870967 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:30.871502 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:30.871507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:30.871509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:30.871623 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:30.871921 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:30.871923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:30.871924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:30.872174 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:30.872301 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:30.873344 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0808 14:43:30.873454 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 14:43:30.873790 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.124:43541
--webserver_interface=127.19.144.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.124.keytab
--principal=kudu/127.19.144.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:33097,127.19.144.125:41803,127.19.144.124:43541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:43:30.874475 10685 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:43:30.875304 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.875010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:43541 (local address 127.0.0.1:34396)
0808 14:43:30.875141 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.875245 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 14:43:30.875931 10633 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
I20260808 14:43:30.876950 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.876709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:43541 (local address 127.0.0.1:34406)
0808 14:43:30.876833 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.876889 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 14:43:30.877137 10685 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:43541: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
I20260808 14:43:30.896673 10607 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } attempt: 1
I20260808 14:43:30.897715 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.897295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:43541 (local address 127.0.0.1:34412)
0808 14:43:30.897494 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.897583 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 14:43:30.898618 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.898361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:43541 (local address 127.0.0.1:34418)
0808 14:43:30.898485 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.898544 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 14:43:30.898841 10607 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:43541: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
I20260808 14:43:30.926529 10685 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } attempt: 1
I20260808 14:43:30.927543 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.927208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:43541 (local address 127.0.0.1:34434)
0808 14:43:30.927375 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.927453 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 14:43:30.928439 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.928188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:43541 (local address 127.0.0.1:34448)
0808 14:43:30.928312 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.928376 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 14:43:30.928645 10685 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:43541: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
I20260808 14:43:30.974087 10607 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } attempt: 2
I20260808 14:43:30.975106 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.974694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:43541 (local address 127.0.0.1:34462)
0808 14:43:30.974903 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.974996 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 14:43:30.975991 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.975727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:43541 (local address 127.0.0.1:34466)
0808 14:43:30.975856 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.975915 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 14:43:30.976188 10607 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:43541: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
I20260808 14:43:30.996824 10685 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } attempt: 2
I20260808 14:43:30.997807 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.997438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:43541 (local address 127.0.0.1:34482)
0808 14:43:30.997597 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.997706 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 14:43:30.998633 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:30.998403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:43541 (local address 127.0.0.1:34486)
0808 14:43:30.998525 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:30.998580 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 14:43:30.998773 10685 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:43541: Network error: Client connection negotiation failed: client connection to 127.19.144.124:43541: connect: Connection refused (error 111)
W20260808 14:43:30.999315 10691 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 14:43:30.999656 10691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:30.999718 10691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:30.999805 10691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:31.004072 10691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:31.004175 10691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:31.004216 10691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:31.004278 10691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:31.004314 10691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:31.010157 10691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:33097,127.19.144.125:41803,127.19.144.124:43541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.124.keytab
--principal=kudu/127.19.144.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.124:43541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.144.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:31.011497 10691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:31.012629 10691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:31.021952 10697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:31.021940 10698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:31.022215 10691 server_base.cc:1034] running on GCE node
W20260808 14:43:31.021940 10700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:31.023113 10691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:31.024351 10691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:31.025533 10691 hybrid_clock.cc:648] HybridClock initialized: now 1786200211025510 us; error 37 us; skew 500 ppm
Aug 08 14:43:31 dist-test-slave-p3t5 krb5kdc[10523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200211, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:31.028657 10691 init.cc:377] Logged in from keytab as kudu/127.19.144.124@KRBTEST.COM (short username kudu)
I20260808 14:43:31.029742 10691 webserver.cc:460] Webserver started at http://127.19.144.124:39279/ using document root <none> and password file <none>
I20260808 14:43:31.030333 10691 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:31.030397 10691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:31.030573 10691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:31.032728 10691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/data/instance:
uuid: "09a2a24d2fc74a069e90ee762b5d379c"
format_stamp: "Formatted at 2026-08-08 14:43:31 on dist-test-slave-p3t5"
server_key: "a06843176c80bfb28faf6c3cd16ed1b0"
server_key_iv: "1409c959008484669b4b42e1f9c8252e"
server_key_version: "encryptionkey@0"
I20260808 14:43:31.033286 10691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/wal/instance:
uuid: "09a2a24d2fc74a069e90ee762b5d379c"
format_stamp: "Formatted at 2026-08-08 14:43:31 on dist-test-slave-p3t5"
server_key: "a06843176c80bfb28faf6c3cd16ed1b0"
server_key_iv: "1409c959008484669b4b42e1f9c8252e"
server_key_version: "encryptionkey@0"
I20260808 14:43:31.037410 10691 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 14:43:31.040437 10707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:31.041987 10691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 14:43:31.042155 10691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/wal
uuid: "09a2a24d2fc74a069e90ee762b5d379c"
format_stamp: "Formatted at 2026-08-08 14:43:31 on dist-test-slave-p3t5"
server_key: "a06843176c80bfb28faf6c3cd16ed1b0"
server_key_iv: "1409c959008484669b4b42e1f9c8252e"
server_key_version: "encryptionkey@0"
I20260808 14:43:31.042300 10691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:31.058691 10691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:31.062110 10691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:31.062348 10691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:31.070185 10691 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.124:43541
I20260808 14:43:31.070538 10761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.124:43541 every 8 connection(s)
I20260808 14:43:31.071391 10691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/data/info.pb
I20260808 14:43:31.072537 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 10691
I20260808 14:43:31.072683 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786199922348708-20033-0/minicluster-data/master-2/wal/instance
I20260808 14:43:31.073041 20033 external_mini_cluster.cc:1428] Setting key 8a42693d46aa9598a5854616fb44fb9a
I20260808 14:43:31.075292 10762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:31.075397 10607 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } attempt: 3
I20260808 14:43:31.080741 10762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:43:31 dist-test-slave-p3t5 krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200210, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for kudu/127.19.144.124@KRBTEST.COM
Aug 08 14:43:31 dist-test-slave-p3t5 krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.124@KRBTEST.COM
I20260808 14:43:31.089404 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:31.075893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:43541 (local address 127.0.0.1:34502)
0808 14:43:31.076092 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:31.076107 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:31.076192 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:31.077050 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:31.077057 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:31.077293 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:31.077421 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:31.077430 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.079178 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.079182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:31.080026 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:31.080040 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.080952 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.080957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:31.081067 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:31.081897 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:31.081915 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:31.083893 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:31.087123 (+ 3230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:31.087130 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:31.087136 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:31.087465 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:31.087895 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:31.087899 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:31.087901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:31.088008 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:31.088829 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:31.088832 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:31.088945 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 14:43:31.089070 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 14:43:31.089227 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260808 14:43:31.089977 10685 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } attempt: 3
Aug 08 14:43:31 dist-test-slave-p3t5 krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200211, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.124@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:43:31.090920 10766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:31.075984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34502 (local address 127.19.144.124:43541)
0808 14:43:31.076336 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.076346 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.076364 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:43:31.076407 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.076410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.076702 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.077558 (+ 856us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.077734 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.078974 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.080182 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.080767 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.081338 (+ 571us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:31.084128 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:31.084135 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:31.084152 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:31.084203 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:31.086998 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.087593 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.087600 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.087606 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.087781 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.088115 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.088118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.088119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.088726 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:31.088869 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:31.089235 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 14:43:31.089769 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":282,"spinlock_wait_cycles":1792,"thread_start_us":134,"threads_started":1}
I20260808 14:43:31.090920 10765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:31.074678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34492 (local address 127.19.144.124:43541)
0808 14:43:31.075191 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.075207 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.075246 (+ 39us) server_negotiation.cc:407] Connection header received
0808 14:43:31.076270 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.076298 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.076699 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.077056 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.077955 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.078914 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.080210 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.080755 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.080943 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:31.084244 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:31.084250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:31.084252 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:31.084280 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:31.087018 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.087908 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.087911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.087912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.087989 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.088356 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.088359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.088361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.088720 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:31.088961 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:31.090399 (+ 1438us) server_negotiation.cc:299] Negotiation successful
0808 14:43:31.090518 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":356,"spinlock_wait_cycles":18048,"thread_start_us":173,"threads_started":1}
I20260808 14:43:31.096140 10607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83: Bootstrap starting.
I20260808 14:43:31.097332 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:31.083632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47080 (local address 127.19.144.126:33097)
0808 14:43:31.083775 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.083780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.084259 (+ 479us) server_negotiation.cc:407] Connection header received
0808 14:43:31.084481 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.084486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.084549 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.084642 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.085763 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.086678 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.087800 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.088071 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.088191 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:31.092704 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:31.092708 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:31.092711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:31.092739 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:31.094798 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.095407 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.095410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.095412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.095509 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.096127 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.096131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.096133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.096502 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:31.096631 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:31.097056 (+ 425us) server_negotiation.cc:299] Negotiation successful
0808 14:43:31.097183 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 14:43:31 dist-test-slave-p3t5 krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200210, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.125@KRBTEST.COM for kudu/127.19.144.124@KRBTEST.COM
I20260808 14:43:31.098196 10767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:31.083572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:33097 (local address 127.0.0.1:47080)
0808 14:43:31.084128 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:31.084193 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:31.084372 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:31.084688 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:31.084696 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:31.085263 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:31.085386 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:31.085393 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.086840 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.086846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:31.087682 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:31.087692 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.088210 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.088213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:31.088309 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:31.088906 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:31.088921 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:31.092561 (+ 3640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:31.094925 (+ 2364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:31.094938 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:31.094944 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:31.095285 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:31.095626 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:31.095629 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:31.095631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:31.095732 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:31.096641 (+ 909us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:31.096648 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:31.096789 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 14:43:31.097748 (+ 959us) client_negotiation.cc:240] Negotiation successful
0808 14:43:31.097988 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":138,"threads_started":1}
I20260808 14:43:31.099476 10607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:31.099906 10762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:43:31.100438 10607 log.cc:824] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:43:31 dist-test-slave-p3t5 krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:43:31.103195 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:31.091173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:43541 (local address 127.0.0.1:34506)
0808 14:43:31.091474 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:31.091490 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:31.091582 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:31.092154 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:31.092157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:31.092357 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:31.092491 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:31.092504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.094016 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.094021 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:31.094770 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:31.094782 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.095497 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.095501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:31.095601 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:31.096083 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:31.096107 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:31.097839 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:31.100694 (+ 2855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:31.100697 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:31.100699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:31.101043 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:31.101983 (+ 940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:31.101989 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:31.101993 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:31.102107 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:31.102629 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:31.102632 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:31.102738 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 14:43:31.102853 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 14:43:31.102996 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206}
I20260808 14:43:31.103828 10768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:31.091489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34506 (local address 127.19.144.124:43541)
0808 14:43:31.091828 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.091837 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.091851 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:43:31.091900 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.091903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.091949 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.092624 (+ 675us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.092742 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.093872 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.094948 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.095345 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.095489 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:31.098019 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:31.098024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:31.098027 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:31.098053 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:31.100591 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.101721 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.101728 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.101733 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.101845 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.102218 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.102221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.102223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.102478 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:31.102581 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:31.103492 (+ 911us) server_negotiation.cc:299] Negotiation successful
0808 14:43:31.103635 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":139,"threads_started":1}
I20260808 14:43:31.104022 10607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83: No bootstrap required, opened a new log
Aug 08 14:43:31 dist-test-slave-p3t5 krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200211, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.124@KRBTEST.COM for kudu/127.19.144.125@KRBTEST.COM
I20260808 14:43:31.107518 10685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2: Bootstrap starting.
I20260808 14:43:31.109051 10770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:31.094311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47086 (local address 127.19.144.126:33097)
0808 14:43:31.095716 (+ 1405us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.095734 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.095751 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:43:31.095796 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.095801 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.095844 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.095970 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.097592 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.098566 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.099950 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.100420 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.100546 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:31.104282 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:31.104287 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:31.104292 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:31.104324 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:31.106477 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.107037 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.107041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.107043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.107148 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.107472 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.107475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.107477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.107819 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:31.107995 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:31.108659 (+ 664us) server_negotiation.cc:299] Negotiation successful
0808 14:43:31.108810 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1316,"thread_start_us":82,"threads_started":1}
I20260808 14:43:31.109408 10607 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } }
I20260808 14:43:31.109741 10607 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7f7f51ae73f445584e6830410982c83, State: Initialized, Role: FOLLOWER
I20260808 14:43:31.110164 10607 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } }
I20260808 14:43:31.110767 10772 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } } }
I20260808 14:43:31.111055 10772 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:43:31.110850 10685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:31.111886 10607 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:31.111889 10685 log.cc:824] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:31.112766 10767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:31.100908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:41803 (local address 127.0.0.1:53610)
0808 14:43:31.101241 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:31.101257 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:31.101340 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:31.101574 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:31.101577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:31.101788 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:31.101883 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:31.101889 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.102980 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.102983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:31.103827 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:31.103839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.104589 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.104593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:31.104680 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:31.105086 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:31.105101 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:31.106765 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:31.109709 (+ 2944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:31.109713 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:31.109715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:31.109988 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:31.111460 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:31.111464 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:31.111466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:31.111573 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:31.112144 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:31.112148 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:31.112266 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 14:43:31.112398 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 14:43:31.112558 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248}
I20260808 14:43:31.113445 10689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:31.101212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53610 (local address 127.19.144.125:41803)
0808 14:43:31.101335 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.101340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.101354 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:43:31.101425 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.101428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.101477 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.101560 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.102018 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.102833 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.103978 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.104328 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.104463 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:31.106948 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:31.106953 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:31.106956 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:31.106983 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:31.109265 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.111213 (+ 1948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.111216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.111219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.111320 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.111693 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.111697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.111699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.111987 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:31.112093 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:31.113132 (+ 1039us) server_negotiation.cc:299] Negotiation successful
0808 14:43:31.113273 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 14:43:31.114171 10762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:43:31.114246 10685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2: No bootstrap required, opened a new log
Aug 08 14:43:31 dist-test-slave-p3t5 krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.125@KRBTEST.COM
I20260808 14:43:31.118939 10685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } }
I20260808 14:43:31.119199 10685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71ba32eefe1b4b10a579a0d9298ec3a2, State: Initialized, Role: FOLLOWER
I20260808 14:43:31.119562 10685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } }
Aug 08 14:43:31 dist-test-slave-p3t5 krb5kdc[10523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200211, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.124@KRBTEST.COM for kudu/127.19.144.124@KRBTEST.COM
I20260808 14:43:31.121223 10685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:31.117568 10537 master_runner.cc:186] Error getting master registration for 127.19.144.125:41803: OK, instance_id { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" instance_seqno: 1786200210827767 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 14:43:31.122473 10537 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 14:43:31.124776 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:31.110936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53624 (local address 127.19.144.125:41803)
0808 14:43:31.112468 (+ 1532us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.112479 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.112496 (+ 17us) server_negotiation.cc:407] Connection header received
0808 14:43:31.112531 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.112536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.112583 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.113459 (+ 876us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.113555 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.114359 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.115855 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.116209 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.119428 (+ 3219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:31.119537 (+ 109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:31.119540 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:31.119543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:31.119559 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:31.121388 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.122211 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.122214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.122216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.122329 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.122673 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.122676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.122678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.122910 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:31.123859 (+ 949us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:31.123953 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 14:43:31.124100 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1437,"thread_start_us":107,"threads_started":1}
I20260808 14:43:31.124995 10768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:31.114893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34520 (local address 127.19.144.124:43541)
0808 14:43:31.115467 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.115471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.115483 (+ 12us) server_negotiation.cc:407] Connection header received
0808 14:43:31.115514 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.115517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.115554 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.115713 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.116145 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.116924 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.118150 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.118575 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.118653 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:31.121216 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:31.121221 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:31.121224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:31.121248 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:31.123177 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.123672 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.123675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.123677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.123762 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.124057 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.124060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.124062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.124267 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:31.124342 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:31.124709 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 14:43:31.124835 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499}
I20260808 14:43:31.125511 10767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:31.114693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:43541 (local address 127.0.0.1:34520)
0808 14:43:31.114997 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:31.115013 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:31.115096 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:31.115701 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:31.115705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:31.115900 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:31.116022 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:31.116031 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.117073 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.117076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:31.117993 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:31.118009 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.118798 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.118802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:31.118884 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:31.119428 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:31.119444 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:31.121064 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:31.123297 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:31.123300 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:31.123303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:31.123565 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:31.123867 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:31.123870 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:31.123872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:31.123959 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:31.124423 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:31.124427 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:31.124539 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 14:43:31.125194 (+ 655us) client_negotiation.cc:240] Negotiation successful
0808 14:43:31.125338 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238}
I20260808 14:43:31.125572 10787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } } }
I20260808 14:43:31.125746 10787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:43:31.128818 10762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c: Bootstrap starting.
I20260808 14:43:31.129257 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 14:43:31.129608 10799 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:43:31.129717 10799 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 14:43:31.129791 10786 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:43:31.129892 10786 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:43:31.130771 10580 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a7f7f51ae73f445584e6830410982c83"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47086
I20260808 14:43:31.131021 10580 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 14:43:31.131145 10580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:31.131678 10762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:31.132586 10762 log.cc:824] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:31.132609 10580 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } }
I20260808 14:43:31.134259 10580 leader_election.cc:288] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [CANDIDATE]: Term 1 election: Requested vote from peers 71ba32eefe1b4b10a579a0d9298ec3a2 (127.19.144.125:41803), 09a2a24d2fc74a069e90ee762b5d379c (127.19.144.124:43541)
I20260808 14:43:31.134280 10658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a7f7f51ae73f445584e6830410982c83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2"
I20260808 14:43:31.134513 10658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:31.131736 10614 master_runner.cc:186] Error getting master registration for 127.19.144.124:43541: OK, instance_id { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" instance_seqno: 1786200211067899 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 14:43:31.135656 10614 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 14:43:31.135915 10658 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7f7f51ae73f445584e6830410982c83 in term 1.
I20260808 14:43:31.136451 10553 leader_election.cc:302] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71ba32eefe1b4b10a579a0d9298ec3a2, a7f7f51ae73f445584e6830410982c83; no voters:
I20260808 14:43:31.136808 10772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:31.137063 10772 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [term 1 LEADER]: Becoming Leader. State: Replica: a7f7f51ae73f445584e6830410982c83, State: Running, Role: LEADER
I20260808 14:43:31.137454 10772 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } }
I20260808 14:43:31.140538 10807 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7f7f51ae73f445584e6830410982c83. Latest consensus state: current_term: 1 leader_uuid: "a7f7f51ae73f445584e6830410982c83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } } }
I20260808 14:43:31.140799 10807 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:31.141479 10809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:31.143980 10762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c: No bootstrap required, opened a new log
I20260808 14:43:31.149408 10762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } }
I20260808 14:43:31.149778 10762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09a2a24d2fc74a069e90ee762b5d379c, State: Initialized, Role: FOLLOWER
I20260808 14:43:31.150177 10762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } }
I20260808 14:43:31.150892 10814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } } }
I20260808 14:43:31.151253 10814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:43:31.151728 10762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:31.151772 10735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a7f7f51ae73f445584e6830410982c83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "09a2a24d2fc74a069e90ee762b5d379c"
I20260808 14:43:31.152030 10735 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:31.153687 10735 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7f7f51ae73f445584e6830410982c83 in term 1.
I20260808 14:43:31.154531 10809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:31.166021 10735 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c [term 1 FOLLOWER]: Refusing update from remote peer a7f7f51ae73f445584e6830410982c83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260808 14:43:31.166182 10827 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:43:31.166265 10827 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:43:31.166949 10807 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:43:31.171422 10814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7f7f51ae73f445584e6830410982c83. Latest consensus state: current_term: 1 leader_uuid: "a7f7f51ae73f445584e6830410982c83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } } }
I20260808 14:43:31.171686 10814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:43:31.172845 10658 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2 [term 1 FOLLOWER]: Refusing update from remote peer a7f7f51ae73f445584e6830410982c83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:43:31.173565 10807 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:43:31.180114 10768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:31.142184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34526 (local address 127.19.144.124:43541)
0808 14:43:31.142339 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.142344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.157866 (+ 15522us) server_negotiation.cc:407] Connection header received
0808 14:43:31.158167 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.158170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.158220 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.158271 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.160804 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.161638 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.168512 (+ 6874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.168878 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.168957 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:31.173763 (+ 4806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:31.173780 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:31.173784 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:31.173820 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:31.175596 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.178558 (+ 2962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.178563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.178564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.178665 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.179019 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.179023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.179025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.179268 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:31.179398 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:31.179790 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 14:43:31.179943 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 14:43:31.180172 10770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:31.142786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47092 (local address 127.19.144.126:33097)
0808 14:43:31.145788 (+ 3002us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.145793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.159525 (+ 13732us) server_negotiation.cc:407] Connection header received
0808 14:43:31.159688 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.159691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.159739 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.159804 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.166422 (+ 6618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.167272 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.169787 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.170170 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.170251 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:31.176152 (+ 5901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:31.176156 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:31.176160 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:31.176195 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:31.177995 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.178562 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.178567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.178569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.178693 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.179075 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.179079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.179082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.179327 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:31.179474 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:31.179840 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 14:43:31.179997 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2866}
I20260808 14:43:31.182540 10787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7f7f51ae73f445584e6830410982c83. Latest consensus state: current_term: 1 leader_uuid: "a7f7f51ae73f445584e6830410982c83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } } }
I20260808 14:43:31.182698 10787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:43:31.188733 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:31.143303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53640 (local address 127.19.144.125:41803)
0808 14:43:31.143449 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.143454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.172387 (+ 28933us) server_negotiation.cc:407] Connection header received
0808 14:43:31.172518 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.172525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.172590 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.172669 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.175344 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.176154 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.182040 (+ 5886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:31.182482 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:31.182600 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:31.184570 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:31.184575 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:31.184578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:31.184610 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:31.186588 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.187171 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.187175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.187177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.187273 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:31.187601 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:31.187604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:31.187606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:31.187851 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:31.187980 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:31.188390 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 14:43:31.188545 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 14:43:31.192895 10772 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7f7f51ae73f445584e6830410982c83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } } }
I20260808 14:43:31.193058 10772 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:31.193348 10826 mvcc.cc:205] Tried to move back new op lower bound from 7316276064907218944 to 7316276064823533568. Current Snapshot: MvccSnapshot[applied={T|T < 7316276064907218944}]
I20260808 14:43:31.194031 10814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7f7f51ae73f445584e6830410982c83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } } }
I20260808 14:43:31.194278 10814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:43:31.195410 10829 mvcc.cc:205] Tried to move back new op lower bound from 7316276064907218944 to 7316276064823533568. Current Snapshot: MvccSnapshot[applied={T|T < 7316276064907218944}]
I20260808 14:43:31.203616 10807 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7f7f51ae73f445584e6830410982c83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } } }
I20260808 14:43:31.203765 10807 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:31.205879 10787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7f7f51ae73f445584e6830410982c83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f7f51ae73f445584e6830410982c83" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 33097 } } peers { permanent_uuid: "71ba32eefe1b4b10a579a0d9298ec3a2" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 41803 } } peers { permanent_uuid: "09a2a24d2fc74a069e90ee762b5d379c" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 43541 } } }
I20260808 14:43:31.206008 10787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:43:31.206907 10809 catalog_manager.cc:1347] Generated new cluster ID: 99b56146a2a84ae49334b485cbcac282
I20260808 14:43:31.206981 10809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:31.218802 10831 mvcc.cc:205] Tried to move back new op lower bound from 7316276064907218944 to 7316276064823533568. Current Snapshot: MvccSnapshot[applied={T|T < 7316276064907218944 or (T in {7316276064907218944})}]
I20260808 14:43:31.266999 10809 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:31.268550 10809 catalog_manager.cc:1504] Loading token signing keys...
I20260808 14:43:31.300078 10809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7f7f51ae73f445584e6830410982c83: Generated new TSK 0
I20260808 14:43:31.301082 10809 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 14:43:31.338282 10770 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:31.334801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47098 (local address 127.19.144.126:33097)
0808 14:43:31.334970 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.334976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.336977 (+ 2001us) server_negotiation.cc:407] Connection header received
0808 14:43:31.337217 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.337221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.337281 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.337342 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.338132 (+ 790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47098: BlockingRecv error: recv got EOF from 127.0.0.1:47098 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 14:43:31.339815 10768 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:31.335635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34536 (local address 127.19.144.124:43541)
0808 14:43:31.335758 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.335763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.338839 (+ 3076us) server_negotiation.cc:407] Connection header received
0808 14:43:31.339015 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.339019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.339073 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.339127 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.339679 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34536: BlockingRecv error: recv got EOF from 127.0.0.1:34536 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 14:43:31.341424 10773 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:31.336407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53646 (local address 127.19.144.125:41803)
0808 14:43:31.336558 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.336563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.340299 (+ 3736us) server_negotiation.cc:407] Connection header received
0808 14:43:31.340533 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.340537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.340592 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.340680 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.341297 (+ 617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53646: BlockingRecv error: recv got EOF from 127.0.0.1:53646 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 14:43:31.346516 10773 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:31.344710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53650 (local address 127.19.144.125:41803)
0808 14:43:31.344820 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.344824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.345396 (+ 572us) server_negotiation.cc:407] Connection header received
0808 14:43:31.345672 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.345677 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.345753 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.345810 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.346403 (+ 593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53650: BlockingRecv error: recv got EOF from 127.0.0.1:53650 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 14:43:31.351732 10773 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:31.350206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53654 (local address 127.19.144.125:41803)
0808 14:43:31.350308 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.350312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.350777 (+ 465us) server_negotiation.cc:407] Connection header received
0808 14:43:31.350956 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.350960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.351011 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.351063 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.351623 (+ 560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53654: BlockingRecv error: recv got EOF from 127.0.0.1:53654 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 14:43:31.357748 10773 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:31.356072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53668 (local address 127.19.144.125:41803)
0808 14:43:31.356175 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.356179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.356754 (+ 575us) server_negotiation.cc:407] Connection header received
0808 14:43:31.356941 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.356946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.356998 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.357051 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.357589 (+ 538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53668: BlockingRecv error: recv got EOF from 127.0.0.1:53668 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 14:43:31.371356 10773 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:31.364510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53674 (local address 127.19.144.125:41803)
0808 14:43:31.364666 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.364671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.370144 (+ 5473us) server_negotiation.cc:407] Connection header received
0808 14:43:31.370424 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.370428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.370505 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.370566 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.371224 (+ 658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53674: BlockingRecv error: recv got EOF from 127.0.0.1:53674 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260808 14:43:31.383270 10773 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:31.381970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53678 (local address 127.19.144.125:41803)
0808 14:43:31.382098 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.382104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.382336 (+ 232us) server_negotiation.cc:407] Connection header received
0808 14:43:31.382508 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.382511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.382565 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.382657 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.383127 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53678: BlockingRecv error: recv got EOF from 127.0.0.1:53678 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 14:43:31.403106 10773 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:31.401775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53692 (local address 127.19.144.125:41803)
0808 14:43:31.402059 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.402065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.402083 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:43:31.402280 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.402283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.402347 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.402444 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.402971 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53692: BlockingRecv error: recv got EOF from 127.0.0.1:53692 (error 108)
Metrics: {"server-negotiator.queue_time_us":175}
W20260808 14:43:31.438720 10773 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:31.437515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53708 (local address 127.19.144.125:41803)
0808 14:43:31.437716 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.437720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.437808 (+ 88us) server_negotiation.cc:407] Connection header received
0808 14:43:31.437974 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.437977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.438030 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.438126 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.438560 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53708: BlockingRecv error: recv got EOF from 127.0.0.1:53708 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 14:43:31.506392 10773 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:31.505200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53710 (local address 127.19.144.125:41803)
0808 14:43:31.505413 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.505416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.505437 (+ 21us) server_negotiation.cc:407] Connection header received
0808 14:43:31.505588 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.505590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.505660 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.505733 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.506260 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53710: BlockingRecv error: recv got EOF from 127.0.0.1:53710 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
W20260808 14:43:31.638034 10773 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:31.636655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53712 (local address 127.19.144.125:41803)
0808 14:43:31.636868 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.636874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.636996 (+ 122us) server_negotiation.cc:407] Connection header received
0808 14:43:31.637193 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.637196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.637246 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.637348 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.637904 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53712: BlockingRecv error: recv got EOF from 127.0.0.1:53712 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
W20260808 14:43:31.898140 10773 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:31.896694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53722 (local address 127.19.144.125:41803)
0808 14:43:31.896868 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:31.896873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:31.897001 (+ 128us) server_negotiation.cc:407] Connection header received
0808 14:43:31.897207 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:31.897210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:31.897266 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:31.897366 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:31.897985 (+ 619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53722: BlockingRecv error: recv got EOF from 127.0.0.1:53722 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 14:43:32.131085 10799 catalog_manager.cc:1259] Loaded cluster ID: 99b56146a2a84ae49334b485cbcac282
I20260808 14:43:32.131177 10799 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2: loading cluster ID for follower catalog manager: success
I20260808 14:43:32.132895 10799 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2: acquiring CA information for follower catalog manager: success
I20260808 14:43:32.134162 10799 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 71ba32eefe1b4b10a579a0d9298ec3a2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 14:43:32.168084 10827 catalog_manager.cc:1259] Loaded cluster ID: 99b56146a2a84ae49334b485cbcac282
I20260808 14:43:32.168166 10827 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c: loading cluster ID for follower catalog manager: success
I20260808 14:43:32.170074 10827 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c: acquiring CA information for follower catalog manager: success
I20260808 14:43:32.171286 10827 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 09a2a24d2fc74a069e90ee762b5d379c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 14:43:32.416712 10902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:32.412593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53730 (local address 127.19.144.125:41803)
0808 14:43:32.412907 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:32.412911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:32.412929 (+ 18us) server_negotiation.cc:407] Connection header received
0808 14:43:32.413008 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:32.413010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:32.413141 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:32.413250 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:43:32.413686 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:32.414329 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:32.415262 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:32.415510 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:32.415595 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:32.415914 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:43:32.416368 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:43:32.416465 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:32.416528 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 14:43:32.416582 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":128,"threads_started":1}
I20260808 14:43:32.424259 10908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:32.420109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34546 (local address 127.19.144.124:43541)
0808 14:43:32.420349 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:32.420353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:32.420366 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:43:32.420511 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:32.420514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:32.420639 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:32.420792 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:43:32.421003 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:32.421677 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:32.422759 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:32.422990 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:32.423074 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:32.423374 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:43:32.423908 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:43:32.424006 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:32.424073 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 14:43:32.424127 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":94,"threads_started":1}
I20260808 14:43:32.432536 10902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:32.428174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53742 (local address 127.19.144.125:41803)
0808 14:43:32.428311 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:32.428315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:32.428328 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:43:32.428488 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:32.428491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:32.428543 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:32.428623 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:43:32.428993 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:32.429974 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:32.430980 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:32.431379 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:32.431523 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:32.431935 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:43:32.432063 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:43:32.432141 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:32.432273 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 14:43:32.432349 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 14:43:32.432732 10908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:32.427882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34562 (local address 127.19.144.124:43541)
0808 14:43:32.428021 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:32.428027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:32.428581 (+ 554us) server_negotiation.cc:407] Connection header received
0808 14:43:32.428769 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:32.428772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:32.428828 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:32.428928 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:43:32.429203 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:32.429788 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:32.431094 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:32.431476 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:32.431579 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:32.432099 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:43:32.432196 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:43:32.432299 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:32.432499 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 14:43:32.432562 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 14:43:32.433248 10915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:32.427738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47102 (local address 127.19.144.126:33097)
0808 14:43:32.428028 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:32.428033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:32.428266 (+ 233us) server_negotiation.cc:407] Connection header received
0808 14:43:32.428392 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:32.428395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:32.428618 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:32.428736 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:43:32.429040 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:32.429977 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:32.431158 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:32.431389 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:32.431508 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:32.432200 (+ 692us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:43:32.432602 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:43:32.432691 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:32.433020 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 14:43:32.433089 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":74,"threads_started":1}
I20260808 14:43:32.436602 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 10537
I20260808 14:43:32.445717 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 10614
I20260808 14:43:32.454126 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 10691
2026-08-08T14:43:32Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3317 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 14:43:32 dist-test-slave-p3t5 krb5kdc[10920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 14:43:32 dist-test-slave-p3t5 krb5kdc[10920](info): set up 2 sockets
Aug 08 14:43:32 dist-test-slave-p3t5 krb5kdc[10920](info): commencing operation
krb5kdc: starting...
W20260808 14:43:33.523434 20033 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/test-user.keytab.
Aug 08 14:43:33 dist-test-slave-p3t5 krb5kdc[10920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T14:43:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T14:43:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.144.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
Entry for principal kudu/127.19.144.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab.
I20260808 14:43:33.664672 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:34025
--webserver_interface=127.19.144.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:46093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:34025,127.19.144.125:34193,127.19.144.124:40855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 14:43:33.782262 10934 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 14:43:33.782653 10934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:33.782727 10934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:33.782802 10934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:33.787010 10934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:33.787111 10934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:33.787147 10934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:33.787184 10934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:33.787220 10934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:33.792793 10934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:46093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:34025,127.19.144.125:34193,127.19.144.124:40855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.126.keytab
--principal=kudu/127.19.144.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.126:34025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.144.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:33.794111 10934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:33.795281 10934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:33.804447 10943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:33.804426 10939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:33.804489 10934 server_base.cc:1034] running on GCE node
W20260808 14:43:33.804431 10941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:33.805603 10934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:33.806908 10934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:33.808094 10934 hybrid_clock.cc:648] HybridClock initialized: now 1786200213808071 us; error 36 us; skew 500 ppm
Aug 08 14:43:33 dist-test-slave-p3t5 krb5kdc[10920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200213, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:33.816069 10934 init.cc:377] Logged in from keytab as kudu/127.19.144.126@KRBTEST.COM (short username kudu)
I20260808 14:43:33.817130 10934 webserver.cc:460] Webserver started at http://127.19.144.126:33221/ using document root <none> and password file <none>
I20260808 14:43:33.817818 10934 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:33.817889 10934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:33.818140 10934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:33.820322 10934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/data/instance:
uuid: "30aa7112c8af455dbe6f8d2f0ae68550"
format_stamp: "Formatted at 2026-08-08 14:43:33 on dist-test-slave-p3t5"
server_key: "fd28d59d44fd9e5bedf313ab00cc4a4b"
server_key_iv: "825b12bb1b8ca0a414afa15973c1f442"
server_key_version: "encryptionkey@0"
I20260808 14:43:33.820957 10934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/wal/instance:
uuid: "30aa7112c8af455dbe6f8d2f0ae68550"
format_stamp: "Formatted at 2026-08-08 14:43:33 on dist-test-slave-p3t5"
server_key: "fd28d59d44fd9e5bedf313ab00cc4a4b"
server_key_iv: "825b12bb1b8ca0a414afa15973c1f442"
server_key_version: "encryptionkey@0"
I20260808 14:43:33.825111 10934 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 14:43:33.827730 10951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:33.829071 10934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 14:43:33.829241 10934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/wal
uuid: "30aa7112c8af455dbe6f8d2f0ae68550"
format_stamp: "Formatted at 2026-08-08 14:43:33 on dist-test-slave-p3t5"
server_key: "fd28d59d44fd9e5bedf313ab00cc4a4b"
server_key_iv: "825b12bb1b8ca0a414afa15973c1f442"
server_key_version: "encryptionkey@0"
I20260808 14:43:33.829371 10934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:33.844947 10934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:33.848624 10934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:33.848881 10934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:33.856894 10934 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.126:34025
I20260808 14:43:33.857197 11004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.126:34025 every 8 connection(s)
I20260808 14:43:33.858090 10934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/data/info.pb
I20260808 14:43:33.861205 11005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:33.862787 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 10934
I20260808 14:43:33.862929 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-0/wal/instance
I20260808 14:43:33.863251 20033 external_mini_cluster.cc:1428] Setting key d702ffb76ed7b471c7d939812ae66061
I20260808 14:43:33.865938 11005 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.144.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.125@KRBTEST.COM" created.
Aug 08 14:43:33 dist-test-slave-p3t5 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200213, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:43:33.882792 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:33.868632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:34025 (local address 127.0.0.1:43472)
0808 14:43:33.869179 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:33.869222 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:33.870170 (+ 948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:33.870910 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:33.870916 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:33.871420 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:33.871675 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:33.871690 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:33.872876 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:33.872879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:33.873770 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:33.873782 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:33.874504 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:33.874507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:33.874601 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:33.875209 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:33.875227 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:33.877312 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:33.879514 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:33.879526 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:33.879538 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:33.879781 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:33.880123 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:33.880128 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:33.880130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:33.880249 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:33.880913 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:33.880920 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:33.881074 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 14:43:33.881258 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 14:43:33.881699 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":121,"thread_start_us":132,"threads_started":1}
I20260808 14:43:33.882792 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:33.868632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43472 (local address 127.19.144.126:34025)
0808 14:43:33.869174 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:33.869196 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:33.869343 (+ 147us) server_negotiation.cc:407] Connection header received
0808 14:43:33.870507 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:33.870512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:33.870712 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:33.870917 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:33.871836 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:33.872725 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:33.873899 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:33.874330 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:33.874521 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:33.877497 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:33.877503 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:33.877515 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:33.877537 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:33.879396 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:33.879900 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:33.879907 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:33.879909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:33.880027 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:33.880352 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:33.880355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:33.880356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:33.880778 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:33.880926 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:33.881324 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 14:43:33.881619 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":213,"threads_started":1}
I20260808 14:43:33.885751 11005 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:43:33.886552 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:33.886254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:34193 (local address 127.0.0.1:48350)
0808 14:43:33.886401 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:33.886480 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 14:43:33.886852 10954 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.144.125:34193: connect: Connection refused (error 111)
I20260808 14:43:33.887966 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:33.887654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:34193 (local address 127.0.0.1:48362)
0808 14:43:33.887845 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:33.887903 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260808 14:43:33.888140 11005 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.125:34193: Network error: Client connection negotiation failed: client connection to 127.19.144.125:34193: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.125.keytab.
Entry for principal kudu/127.19.144.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.125.keytab.
I20260808 14:43:33.895426 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.125:34193
--webserver_interface=127.19.144.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.125.keytab
--principal=kudu/127.19.144.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:46093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:34025,127.19.144.125:34193,127.19.144.124:40855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:43:33.937454 11005 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } attempt: 1
I20260808 14:43:33.938498 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:33.938147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:34193 (local address 127.0.0.1:48374)
0808 14:43:33.938319 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:33.938397 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 14:43:33.939296 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:33.939050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:34193 (local address 127.0.0.1:48380)
0808 14:43:33.939173 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:33.939231 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 14:43:33.939433 11005 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.125:34193: Network error: Client connection negotiation failed: client connection to 127.19.144.125:34193: connect: Connection refused (error 111)
I20260808 14:43:34.007619 11005 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } attempt: 2
I20260808 14:43:34.008651 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.008304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:34193 (local address 127.0.0.1:48396)
0808 14:43:34.008495 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.008565 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260808 14:43:34.009473 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.009245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:34193 (local address 127.0.0.1:48398)
0808 14:43:34.009359 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.009415 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 14:43:34.009598 11005 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.125:34193: Network error: Client connection negotiation failed: client connection to 127.19.144.125:34193: connect: Connection refused (error 111)
W20260808 14:43:34.020810 11012 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 14:43:34.021193 11012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:34.021261 11012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:34.021343 11012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:34.025583 11012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:34.025735 11012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:34.025775 11012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:34.025837 11012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:34.025874 11012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:34.031376 11012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:46093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:34025,127.19.144.125:34193,127.19.144.124:40855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.125.keytab
--principal=kudu/127.19.144.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.125:34193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.144.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:34.032656 11012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:34.033913 11012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 14:43:34.043051 11018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:34.043256 11012 server_base.cc:1034] running on GCE node
W20260808 14:43:34.043076 11017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:34.043185 11020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:34.044080 11012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:34.045408 11012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:34.046629 11012 hybrid_clock.cc:648] HybridClock initialized: now 1786200214046604 us; error 49 us; skew 500 ppm
Aug 08 14:43:34 dist-test-slave-p3t5 krb5kdc[10920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200214, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:34.049782 11012 init.cc:377] Logged in from keytab as kudu/127.19.144.125@KRBTEST.COM (short username kudu)
I20260808 14:43:34.050768 11012 webserver.cc:460] Webserver started at http://127.19.144.125:45319/ using document root <none> and password file <none>
I20260808 14:43:34.051376 11012 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:34.051450 11012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:34.051688 11012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:34.054008 11012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/data/instance:
uuid: "7118d96501eb4c42be2cbd75936907db"
format_stamp: "Formatted at 2026-08-08 14:43:34 on dist-test-slave-p3t5"
server_key: "e771d4d413c413ff8a3a31fe49db4dbd"
server_key_iv: "474808def2a461db945cbc453dbb5c54"
server_key_version: "encryptionkey@0"
I20260808 14:43:34.054634 11012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/wal/instance:
uuid: "7118d96501eb4c42be2cbd75936907db"
format_stamp: "Formatted at 2026-08-08 14:43:34 on dist-test-slave-p3t5"
server_key: "e771d4d413c413ff8a3a31fe49db4dbd"
server_key_iv: "474808def2a461db945cbc453dbb5c54"
server_key_version: "encryptionkey@0"
I20260808 14:43:34.058720 11012 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 14:43:34.061738 11027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:34.063175 11012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 14:43:34.063325 11012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/wal
uuid: "7118d96501eb4c42be2cbd75936907db"
format_stamp: "Formatted at 2026-08-08 14:43:34 on dist-test-slave-p3t5"
server_key: "e771d4d413c413ff8a3a31fe49db4dbd"
server_key_iv: "474808def2a461db945cbc453dbb5c54"
server_key_version: "encryptionkey@0"
I20260808 14:43:34.063448 11012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:34.095358 11012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:34.098919 11012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:34.099172 11012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:34.100798 11005 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } attempt: 3
I20260808 14:43:34.101778 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.101369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:34193 (local address 127.0.0.1:48406)
0808 14:43:34.101558 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.101670 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 14:43:34.102747 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.102479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:34193 (local address 127.0.0.1:48414)
0808 14:43:34.102614 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.102672 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.125:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 14:43:34.102939 11005 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.125:34193: Network error: Client connection negotiation failed: client connection to 127.19.144.125:34193: connect: Connection refused (error 111)
I20260808 14:43:34.107291 11012 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.125:34193
I20260808 14:43:34.107653 11084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.125:34193 every 8 connection(s)
I20260808 14:43:34.108366 11012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/data/info.pb
I20260808 14:43:34.111698 11085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:34.114156 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 11012
I20260808 14:43:34.114327 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-1/wal/instance
I20260808 14:43:34.114625 20033 external_mini_cluster.cc:1428] Setting key cd5bfefe39ee39d5a0101bd463f16797
I20260808 14:43:34.116385 11085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.144.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.144.124@KRBTEST.COM" created.
Aug 08 14:43:34 dist-test-slave-p3t5 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200214, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.125@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:43:34.134801 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.119193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43488 (local address 127.19.144.126:34025)
0808 14:43:34.119389 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.119394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.120371 (+ 977us) server_negotiation.cc:407] Connection header received
0808 14:43:34.121416 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.121423 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.121505 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.121670 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.122930 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.123910 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.125151 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.125527 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.125698 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.129370 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:34.129377 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:34.129383 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:34.129420 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:34.131863 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.132643 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.132650 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.132655 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.132788 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.133250 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.133256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.133261 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.133578 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:34.133731 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:34.134468 (+ 737us) server_negotiation.cc:299] Negotiation successful
0808 14:43:34.134631 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 14:43:34.136000 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.119620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:34025 (local address 127.0.0.1:43488)
0808 14:43:34.120170 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.120229 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:34.121150 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:34.121855 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:34.121862 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:34.122237 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:34.122753 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:34.122770 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.124054 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.124057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:34.124968 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:34.124978 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.125757 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.125762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:34.125922 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.126657 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:34.126675 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:34.129143 (+ 2468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:34.132047 (+ 2904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:34.132058 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:34.132080 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:34.132518 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:34.132905 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:34.132908 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:34.132909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:34.133138 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:34.133752 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:34.133758 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:34.134166 (+ 408us) client_negotiation.cc:769] Sending connection context
0808 14:43:34.134355 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 14:43:34.134706 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":183,"threads_started":1}
I20260808 14:43:34.137707 11085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 14:43:34 dist-test-slave-p3t5 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200214, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.125@KRBTEST.COM for kudu/127.19.144.125@KRBTEST.COM
I20260808 14:43:34.148320 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.138335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:34193 (local address 127.0.0.1:48422)
0808 14:43:34.138519 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.138533 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:34.138617 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:34.139216 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:34.139219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:34.139407 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:34.139495 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:34.139501 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.140641 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.140645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:34.141311 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:34.141323 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.142172 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.142175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:34.142259 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.142776 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:34.142794 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:34.144301 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:34.146557 (+ 2256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:34.146562 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:34.146564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:34.146864 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:34.147185 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:34.147188 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:34.147189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:34.147278 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:34.147772 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:34.147775 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:34.147864 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 14:43:34.147984 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 14:43:34.148130 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260808 14:43:34.148383 11091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.138519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48422 (local address 127.19.144.125:34193)
0808 14:43:34.138861 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.138870 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.138889 (+ 19us) server_negotiation.cc:407] Connection header received
0808 14:43:34.138931 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.138935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.139092 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.139243 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.139609 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.140475 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.141441 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.142019 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.142196 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.144488 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:34.144498 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:34.144537 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:34.144559 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:34.146392 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.146984 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.146990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.146994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.147077 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.147375 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.147378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.147380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.147657 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:34.147813 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:34.148086 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 14:43:34.148225 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":258,"thread_start_us":157,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.144.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.124.keytab.
Entry for principal kudu/127.19.144.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.124.keytab.
I20260808 14:43:34.149502 20033 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
/tmp/dist-test-taskCQkDbp/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.124:40855
--webserver_interface=127.19.144.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.124.keytab
--principal=kudu/127.19.144.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.144.84:46093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.144.126:34025,127.19.144.125:34193,127.19.144.124:40855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 14:43:34.150411 11085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:43:34.151232 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.150954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:40855 (local address 127.0.0.1:57982)
0808 14:43:34.151083 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.151165 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 14:43:34.151446 11028 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
I20260808 14:43:34.152261 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.152042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:40855 (local address 127.0.0.1:57994)
0808 14:43:34.152151 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.152196 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 14:43:34.152436 11085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:40855: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
I20260808 14:43:34.201720 11085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } attempt: 1
I20260808 14:43:34.202826 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.202395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:40855 (local address 127.0.0.1:58010)
0808 14:43:34.202579 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.202683 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 14:43:34.203606 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.203368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:40855 (local address 127.0.0.1:58022)
0808 14:43:34.203491 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.203550 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 14:43:34.203739 11085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:40855: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
I20260808 14:43:34.246170 11005 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } attempt: 4
Aug 08 14:43:34 dist-test-slave-p3t5 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200213, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for kudu/127.19.144.125@KRBTEST.COM
I20260808 14:43:34.256158 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.246903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:34193 (local address 127.0.0.1:48426)
0808 14:43:34.247143 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.247161 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:34.247291 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:34.247617 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:34.247620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:34.247838 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:34.247954 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:34.247960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.249140 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.249145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:34.249753 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:34.249762 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.250288 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.250291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:34.250414 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.250991 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:34.251013 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:34.252578 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:34.254524 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:34.254529 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:34.254531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:34.254753 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:34.255074 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:34.255077 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:34.255079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:34.255171 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:34.255578 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:34.255581 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:34.255696 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 14:43:34.255808 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 14:43:34.255970 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260808 14:43:34.256158 11091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.247020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48426 (local address 127.19.144.125:34193)
0808 14:43:34.247172 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.247179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.247219 (+ 40us) server_negotiation.cc:407] Connection header received
0808 14:43:34.247429 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.247432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.247492 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.247590 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.248092 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.249005 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.249899 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.250159 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.250279 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.252739 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:34.252743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:34.252745 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:34.252772 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:34.254396 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.254863 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.254867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.254869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.254960 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.255264 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.255269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.255271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.255474 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:34.255587 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:34.255872 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 14:43:34.255999 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 14:43:34.257054 11005 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:43:34.257867 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.257568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:40855 (local address 127.0.0.1:58030)
0808 14:43:34.257733 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.257788 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260808 14:43:34.258492 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.258278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:40855 (local address 127.0.0.1:58042)
0808 14:43:34.258390 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.258437 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 14:43:34.258611 11005 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:40855: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
I20260808 14:43:34.271922 11085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } attempt: 2
I20260808 14:43:34.272845 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.272521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:40855 (local address 127.0.0.1:58046)
0808 14:43:34.272684 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.272760 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 14:43:34.273604 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.273384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:40855 (local address 127.0.0.1:58056)
0808 14:43:34.273494 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.273548 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 14:43:34.273427 11092 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 14:43:34.273813 11085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:40855: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
W20260808 14:43:34.273851 11092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 14:43:34.273918 11092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 14:43:34.273998 11092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 14:43:34.278219 11092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 14:43:34.278294 11092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 14:43:34.278333 11092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 14:43:34.278389 11092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 14:43:34.278442 11092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 14:43:34.284123 11092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.144.84:46093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.144.126:34025,127.19.144.125:34193,127.19.144.124:40855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/krb5kdc/kudu_127.19.144.124.keytab
--principal=kudu/127.19.144.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.144.124:40855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.144.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 13:56:08 UTC on ccycloud.quasar-tfdlcx.root.comops.site
build id 4574926
I20260808 14:43:34.285364 11092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 14:43:34.286550 11092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 14:43:34.295688 11092 server_base.cc:1034] running on GCE node
W20260808 14:43:34.295627 11101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:34.295639 11098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 14:43:34.295634 11099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 14:43:34.296553 11092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 14:43:34.297922 11092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 14:43:34.299165 11092 hybrid_clock.cc:648] HybridClock initialized: now 1786200214299124 us; error 53 us; skew 500 ppm
Aug 08 14:43:34 dist-test-slave-p3t5 krb5kdc[10920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200214, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 14:43:34.302294 11092 init.cc:377] Logged in from keytab as kudu/127.19.144.124@KRBTEST.COM (short username kudu)
I20260808 14:43:34.303319 11092 webserver.cc:460] Webserver started at http://127.19.144.124:35513/ using document root <none> and password file <none>
I20260808 14:43:34.303889 11092 fs_manager.cc:359] Metadata directory not provided
I20260808 14:43:34.303951 11092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 14:43:34.304124 11092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 14:43:34.307078 11092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/data/instance:
uuid: "fbb26e1addae462dbe14c8d36afa1a74"
format_stamp: "Formatted at 2026-08-08 14:43:34 on dist-test-slave-p3t5"
server_key: "41192732efd4b3afca8d58c456ab31b3"
server_key_iv: "8fbc1197c2e32c4ed8656e4ff561ef60"
server_key_version: "encryptionkey@0"
I20260808 14:43:34.307922 11092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/wal/instance:
uuid: "fbb26e1addae462dbe14c8d36afa1a74"
format_stamp: "Formatted at 2026-08-08 14:43:34 on dist-test-slave-p3t5"
server_key: "41192732efd4b3afca8d58c456ab31b3"
server_key_iv: "8fbc1197c2e32c4ed8656e4ff561ef60"
server_key_version: "encryptionkey@0"
I20260808 14:43:34.313773 11092 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 14:43:34.317430 11108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 14:43:34.317770 11005 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } attempt: 1
I20260808 14:43:34.318656 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.318328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:40855 (local address 127.0.0.1:58058)
0808 14:43:34.318457 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.318521 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 14:43:34.319360 11092 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 14:43:34.319527 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.319252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:40855 (local address 127.0.0.1:58070)
0808 14:43:34.319372 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.319427 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 14:43:34.319521 11092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/data,/tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/wal
uuid: "fbb26e1addae462dbe14c8d36afa1a74"
format_stamp: "Formatted at 2026-08-08 14:43:34 on dist-test-slave-p3t5"
server_key: "41192732efd4b3afca8d58c456ab31b3"
server_key_iv: "8fbc1197c2e32c4ed8656e4ff561ef60"
server_key_version: "encryptionkey@0"
W20260808 14:43:34.319756 11005 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:40855: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
I20260808 14:43:34.319746 11092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 14:43:34.364641 11092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 14:43:34.364976 11085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } attempt: 3
I20260808 14:43:34.365906 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.365514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:40855 (local address 127.0.0.1:58076)
0808 14:43:34.365740 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.365810 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 14:43:34.366890 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.366501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:40855 (local address 127.0.0.1:58090)
0808 14:43:34.366701 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.366810 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260808 14:43:34.367067 11085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.144.124:40855: Network error: Client connection negotiation failed: client connection to 127.19.144.124:40855: connect: Connection refused (error 111)
I20260808 14:43:34.370193 11092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 14:43:34.370414 11092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 14:43:34.379892 11092 rpc_server.cc:307] RPC server started. Bound to: 127.19.144.124:40855
I20260808 14:43:34.380318 11162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.144.124:40855 every 8 connection(s)
I20260808 14:43:34.381458 11092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/data/info.pb
I20260808 14:43:34.385089 11163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 14:43:34.386951 11005 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } attempt: 2
I20260808 14:43:34.390336 11163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:43:34.391477 20033 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu as pid 11092
I20260808 14:43:34.391601 20033 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCQkDbp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786199922348708-20033-0/minicluster-data/master-2/wal/instance
I20260808 14:43:34.391927 20033 external_mini_cluster.cc:1428] Setting key 6b330d18c5fe9985e0a772ee7c811b99
Aug 08 14:43:34 dist-test-slave-p3t5 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200213, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.126@KRBTEST.COM for kudu/127.19.144.124@KRBTEST.COM
Aug 08 14:43:34 dist-test-slave-p3t5 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200214, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.124@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
Aug 08 14:43:34 dist-test-slave-p3t5 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.124@KRBTEST.COM
I20260808 14:43:34.404224 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.387592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:40855 (local address 127.0.0.1:58100)
0808 14:43:34.387847 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.387865 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:34.387972 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:34.391392 (+ 3420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:34.391396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:34.391621 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:34.391749 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:34.391757 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.393684 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.393691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:34.394297 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:34.394307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.395316 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.395322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:34.395416 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.396239 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:34.396255 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:34.397923 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:34.401440 (+ 3517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:34.401443 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:34.401446 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:34.401852 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:34.402354 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:34.402360 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:34.402364 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:34.402476 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:34.403446 (+ 970us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:34.403450 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:34.403577 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 14:43:34.403773 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 14:43:34.403976 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260808 14:43:34.406078 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.393253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43490 (local address 127.19.144.126:34025)
0808 14:43:34.393374 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.393379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.393392 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:43:34.393429 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.393432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.393478 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.393562 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.394580 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.395272 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.396497 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.396849 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.396929 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.401381 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:34.401388 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:34.401394 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:34.401430 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:34.403291 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.403997 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.404001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.404003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.404112 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.404781 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.404786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.404787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.405100 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:34.405229 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:34.405793 (+ 564us) server_negotiation.cc:299] Negotiation successful
0808 14:43:34.405923 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 14:43:34.407003 11165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.388382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58100 (local address 127.19.144.124:40855)
0808 14:43:34.389122 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.389145 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.389197 (+ 52us) server_negotiation.cc:407] Connection header received
0808 14:43:34.390345 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.390375 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.390935 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.391442 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.391923 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.393470 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.394429 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.395169 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.395718 (+ 549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.398079 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:34.398089 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:34.398107 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:34.398142 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:34.401303 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.402012 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.402022 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.402035 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.402209 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.402612 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.402618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.402623 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.403313 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:34.404800 (+ 1487us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:34.405028 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 14:43:34.405719 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":129,"threads_started":1}
I20260808 14:43:34.406999 11166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.392316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.126:34025 (local address 127.0.0.1:43490)
0808 14:43:34.392769 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.392821 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:34.392936 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:34.393748 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:34.393755 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:34.394288 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:34.394447 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:34.394456 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.395398 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.395401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:34.396351 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:34.396365 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.397135 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.397138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:34.397295 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.398378 (+ 1083us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:34.398397 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:34.400999 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:34.403440 (+ 2441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:34.403454 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:34.403464 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:34.403860 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:34.404518 (+ 658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:34.404523 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:34.404525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:34.404655 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:34.405292 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:34.405302 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:34.405450 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 14:43:34.405721 (+ 271us) client_negotiation.cc:240] Negotiation successful
0808 14:43:34.405932 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":1,"thread_start_us":186,"threads_started":1}
I20260808 14:43:34.408901 11163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:43:34.409891 11168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.392974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58104 (local address 127.19.144.124:40855)
0808 14:43:34.393247 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.393255 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.393405 (+ 150us) server_negotiation.cc:407] Connection header received
0808 14:43:34.393551 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.393554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.393621 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.393753 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.394689 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.395514 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.396816 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.397187 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.397303 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.402141 (+ 4838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:34.402146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:34.402152 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:34.402185 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:34.407009 (+ 4824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.408064 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.408067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.408069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.408169 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.408492 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.408497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.408498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.408731 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:34.408898 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:34.409579 (+ 681us) server_negotiation.cc:299] Negotiation successful
0808 14:43:34.409752 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":497,"server-negotiator.queue_time_us":201,"thread_start_us":104,"threads_started":1}
I20260808 14:43:34.413276 11005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550: Bootstrap starting.
Aug 08 14:43:34 dist-test-slave-p3t5 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200214, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.124@KRBTEST.COM for kudu/127.19.144.125@KRBTEST.COM
I20260808 14:43:34.416531 11005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:34.417671 11005 log.cc:824] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550: Log is configured to *not* fsync() on all Append() calls
Aug 08 14:43:34 dist-test-slave-p3t5 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.126@KRBTEST.COM
I20260808 14:43:34.420286 11005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550: No bootstrap required, opened a new log
I20260808 14:43:34.422501 11091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.409886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48442 (local address 127.19.144.125:34193)
0808 14:43:34.410040 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.410045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.410059 (+ 14us) server_negotiation.cc:407] Connection header received
0808 14:43:34.410101 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.410104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.410150 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.410235 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.411155 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.412007 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.413128 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.413532 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.413667 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.417862 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:34.417865 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:34.417868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:34.417900 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:34.420714 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.421184 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.421188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.421190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.421292 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.421590 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.421595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.421597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.421834 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:34.421947 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:34.422217 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 14:43:34.422334 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 14:43:34.422501 11166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.409535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.125:34193 (local address 127.0.0.1:48442)
0808 14:43:34.409668 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.409683 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:34.409776 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:34.410587 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:34.410593 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:34.410827 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:34.410993 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:34.411005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.412166 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.412169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:34.412963 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:34.412976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.413948 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.413951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:34.414043 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.414819 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:34.414838 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:34.417658 (+ 2820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:34.420851 (+ 3193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:34.420856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:34.420858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:34.421076 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:34.421396 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:34.421400 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:34.421402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:34.421494 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:34.421937 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:34.421940 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:34.422047 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 14:43:34.422159 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 14:43:34.422327 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 14:43:34.423065 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.411255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43492 (local address 127.19.144.126:34025)
0808 14:43:34.411360 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.411364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.411519 (+ 155us) server_negotiation.cc:407] Connection header received
0808 14:43:34.411693 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.411699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.411767 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.411818 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.412684 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.413376 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.414538 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.414888 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.414966 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.418829 (+ 3863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:34.418833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:34.418836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:34.418860 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:34.421037 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.421555 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.421559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.421561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.421680 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.422011 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.422014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.422016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.422227 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:34.422384 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:34.422783 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 14:43:34.422912 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 14:43:34.423681 11163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } has no permanent_uuid. Determining permanent_uuid...
I20260808 14:43:34.424883 11005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } }
I20260808 14:43:34.425143 11005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30aa7112c8af455dbe6f8d2f0ae68550, State: Initialized, Role: FOLLOWER
I20260808 14:43:34.425542 11005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } }
I20260808 14:43:34.426185 11170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } } }
I20260808 14:43:34.426525 11170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:43:34.427181 11005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 14:43:34 dist-test-slave-p3t5 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200214, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.124@KRBTEST.COM for kudu/127.19.144.124@KRBTEST.COM
Aug 08 14:43:34 dist-test-slave-p3t5 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.144.125@KRBTEST.COM
I20260808 14:43:34.434145 11166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.424261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:40855 (local address 127.0.0.1:58108)
0808 14:43:34.424417 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.424429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:34.424519 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:34.424925 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:34.424928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:34.425063 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:34.425178 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:34.425185 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.426384 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.426389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:34.427017 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:34.427027 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.427514 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.427518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:34.427596 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.428066 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:34.428090 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:34.429756 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:34.432448 (+ 2692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:34.432451 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:34.432453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:34.432699 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:34.433022 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:34.433025 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:34.433027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:34.433104 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:34.433580 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:34.433583 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:34.433692 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 14:43:34.433800 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 14:43:34.433969 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
W20260808 14:43:34.434473 11181 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:43:34.434564 11181 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:43:34.434787 11168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.424278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58108 (local address 127.19.144.124:40855)
0808 14:43:34.424730 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.424735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.424748 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:43:34.424782 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.424785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.424824 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.424910 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.425378 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.426263 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.427121 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.427400 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.427504 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.429897 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:34.429900 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:34.429902 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:34.429920 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:34.432321 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.432837 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.432842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.432846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.432927 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.433189 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.433191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.433193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.433453 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:34.433545 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:34.434496 (+ 951us) server_negotiation.cc:299] Negotiation successful
0808 14:43:34.434633 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366}
I20260808 14:43:34.433099 10934 master_runner.cc:186] Error getting master registration for 127.19.144.125:34193: OK, instance_id { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" instance_seqno: 1786200214105061 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 14:43:34.436650 10934 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 14:43:34.438037 11163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74: Bootstrap starting.
I20260808 14:43:34.441095 11163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:34.441821 11091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.424623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48444 (local address 127.19.144.125:34193)
0808 14:43:34.424766 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.424770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.425169 (+ 399us) server_negotiation.cc:407] Connection header received
0808 14:43:34.425354 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.425357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.425398 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.425444 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.426335 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.427032 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.428139 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.428472 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.428557 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.431807 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:34.431814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:34.431819 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:34.431851 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:34.439382 (+ 7531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.439990 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.439994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.439997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.440110 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.440483 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.440487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.440489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.440756 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:34.441000 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:34.441435 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 14:43:34.441581 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 14:43:34.442061 11163 log.cc:824] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:34.444844 11163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74: No bootstrap required, opened a new log
I20260808 14:43:34.445580 20033 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 14:43:34.446900 10978 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "30aa7112c8af455dbe6f8d2f0ae68550"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43492
I20260808 14:43:34.447180 10978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 14:43:34.447324 10978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:34.448776 10978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } }
I20260808 14:43:34.449347 11163 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } }
I20260808 14:43:34.449661 11163 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbb26e1addae462dbe14c8d36afa1a74, State: Initialized, Role: FOLLOWER
I20260808 14:43:34.450023 11163 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } }
I20260808 14:43:34.450642 11184 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } } }
I20260808 14:43:34.450858 11184 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:43:34.451066 10978 leader_election.cc:288] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [CANDIDATE]: Term 1 election: Requested vote from peers 7118d96501eb4c42be2cbd75936907db (127.19.144.125:34193), fbb26e1addae462dbe14c8d36afa1a74 (127.19.144.124:40855)
I20260808 14:43:34.451607 11163 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:34.453831 11135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fbb26e1addae462dbe14c8d36afa1a74"
I20260808 14:43:34.454114 11135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74 [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:34.455652 11135 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30aa7112c8af455dbe6f8d2f0ae68550 in term 1.
I20260808 14:43:34.456177 10952 leader_election.cc:302] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30aa7112c8af455dbe6f8d2f0ae68550, fbb26e1addae462dbe14c8d36afa1a74; no voters:
I20260808 14:43:34.456580 11170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 14:43:34.456816 11170 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [term 1 LEADER]: Becoming Leader. State: Replica: 30aa7112c8af455dbe6f8d2f0ae68550, State: Running, Role: LEADER
I20260808 14:43:34.457178 11170 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } }
I20260808 14:43:34.459188 11170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30aa7112c8af455dbe6f8d2f0ae68550. Latest consensus state: current_term: 1 leader_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } } }
I20260808 14:43:34.459342 11170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:34.465801 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.453917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43496 (local address 127.19.144.126:34025)
0808 14:43:34.454042 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.454047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.454616 (+ 569us) server_negotiation.cc:407] Connection header received
0808 14:43:34.454732 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.454736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.454794 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.454881 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.456607 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.457336 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.458520 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.458823 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.458901 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.461164 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:34.461168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:34.461172 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:34.461203 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:34.463630 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.464232 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.464236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.464238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.464332 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.464659 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.464662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.464664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.464909 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:34.465035 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:34.465449 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 14:43:34.465594 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 14:43:34.467486 11205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 14:43:34.468791 11205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 14:43:34.465868 11092 master_runner.cc:186] Error getting master registration for 127.19.144.125:34193: OK, instance_id { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" instance_seqno: 1786200214105061 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 14:43:34.469528 11092 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 14:43:34.470830 11091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.454212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48450 (local address 127.19.144.125:34193)
0808 14:43:34.454365 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.454369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.454920 (+ 551us) server_negotiation.cc:407] Connection header received
0808 14:43:34.455104 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.455107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.455161 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.455218 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.456347 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.457214 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.458711 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.459076 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.459159 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.461665 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:34.461672 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:34.461677 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:34.461710 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:34.463752 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.464379 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.464383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.464385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.464486 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.464816 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.464819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.464821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.465053 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:34.465180 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:34.470489 (+ 5309us) server_negotiation.cc:299] Negotiation successful
0808 14:43:34.470638 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 14:43:34.472298 11202 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:43:34.472384 11202 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:43:34.477221 11168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.461127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58112 (local address 127.19.144.124:40855)
0808 14:43:34.461250 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.461257 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.465589 (+ 4332us) server_negotiation.cc:407] Connection header received
0808 14:43:34.465930 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.465934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.466007 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.466060 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.467681 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.468434 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.469555 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.469890 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.469991 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.472694 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:34.472699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:34.472702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:34.472729 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:34.474521 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.475264 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.475268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.475270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.475370 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.475940 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.475944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.475945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.476177 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:34.476305 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:34.476918 (+ 613us) server_negotiation.cc:299] Negotiation successful
0808 14:43:34.477051 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 14:43:34.478842 11135 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74 [term 1 FOLLOWER]: Refusing update from remote peer 30aa7112c8af455dbe6f8d2f0ae68550: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:43:34.479610 11203 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:43:34.484376 11184 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30aa7112c8af455dbe6f8d2f0ae68550. Latest consensus state: current_term: 1 leader_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } } }
I20260808 14:43:34.484529 11184 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:43:34.488569 11170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } } }
I20260808 14:43:34.488735 11170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:34.488883 11184 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } } }
I20260808 14:43:34.489012 11184 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:43:34.489312 11208 mvcc.cc:205] Tried to move back new op lower bound from 7316276078495481856 to 7316276078420529152. Current Snapshot: MvccSnapshot[applied={T|T < 7316276078495481856}]
I20260808 14:43:34.490039 11209 mvcc.cc:205] Tried to move back new op lower bound from 7316276078495481856 to 7316276078420529152. Current Snapshot: MvccSnapshot[applied={T|T < 7316276078495481856}]
I20260808 14:43:34.492985 11205 catalog_manager.cc:1347] Generated new cluster ID: 9c8fe5ed4e3f4c4ab0c5ceabdab0f06a
I20260808 14:43:34.493065 11205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 14:43:34.510290 11085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } attempt: 4
I20260808 14:43:34.512272 11205 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 14:43:34.514714 11205 catalog_manager.cc:1504] Loading token signing keys...
Aug 08 14:43:34 dist-test-slave-p3t5 krb5kdc[10920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786200214, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.144.125@KRBTEST.COM for kudu/127.19.144.124@KRBTEST.COM
I20260808 14:43:34.523597 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.510831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.144.124:40855 (local address 127.0.0.1:58128)
0808 14:43:34.510983 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 14:43:34.511002 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 14:43:34.511102 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 14:43:34.511435 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 14:43:34.511438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 14:43:34.511686 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 14:43:34.511802 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:34.511810 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.512874 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.512879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:34.513725 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 14:43:34.513738 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.514504 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.514508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 14:43:34.514594 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.517577 (+ 2983us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 14:43:34.517592 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 14:43:34.519548 (+ 1956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 14:43:34.521844 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:34.521847 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:34.521849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:34.522118 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:34.522457 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 14:43:34.522460 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 14:43:34.522462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 14:43:34.522563 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 14:43:34.523055 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 14:43:34.523058 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 14:43:34.523170 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 14:43:34.523263 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 14:43:34.523415 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 14:43:34.524264 11168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:34.511098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58128 (local address 127.19.144.124:40855)
0808 14:43:34.511212 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.511217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.511233 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:43:34.511275 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.511278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.511329 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.511423 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.511938 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.512682 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.513896 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:34.514225 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:34.514338 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:34.519725 (+ 5387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 14:43:34.519729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 14:43:34.519732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 14:43:34.519761 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 14:43:34.521694 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.522238 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.522241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.522243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.522344 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 14:43:34.522665 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 14:43:34.522668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 14:43:34.522670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 14:43:34.522913 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 14:43:34.523001 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:34.523929 (+ 928us) server_negotiation.cc:299] Negotiation successful
0808 14:43:34.524055 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 14:43:34.527762 11085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db: Bootstrap starting.
I20260808 14:43:34.535051 11085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db: Neither blocks nor log segments found. Creating new log.
I20260808 14:43:34.536054 11085 log.cc:824] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db: Log is configured to *not* fsync() on all Append() calls
I20260808 14:43:34.539008 11205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550: Generated new TSK 0
I20260808 14:43:34.539955 11205 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 14:43:34.546156 11085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db: No bootstrap required, opened a new log
I20260808 14:43:34.555766 11085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } }
I20260808 14:43:34.556115 11085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7118d96501eb4c42be2cbd75936907db, State: Initialized, Role: FOLLOWER
I20260808 14:43:34.556542 11085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } }
I20260808 14:43:34.558368 11085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db [sys.catalog]: configured and running, proceeding with master startup.
I20260808 14:43:34.558521 11057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7118d96501eb4c42be2cbd75936907db"
I20260808 14:43:34.559384 11055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db [term 0 FOLLOWER]: Advancing to term 1
I20260808 14:43:34.561887 11055 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db [term 1 FOLLOWER]: Refusing update from remote peer 30aa7112c8af455dbe6f8d2f0ae68550: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 14:43:34.562834 11203 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 14:43:34.563792 11057 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30aa7112c8af455dbe6f8d2f0ae68550 in term 1.
I20260808 14:43:34.571177 11228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30aa7112c8af455dbe6f8d2f0ae68550. Latest consensus state: current_term: 1 leader_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } } }
I20260808 14:43:34.575214 11228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:43:34.574728 11231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } } }
I20260808 14:43:34.578361 11231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db [sys.catalog]: This master's current role is: FOLLOWER
I20260808 14:43:34.574889 11217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } } }
I20260808 14:43:34.578506 11217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db [sys.catalog]: This master's current role is: FOLLOWER
W20260808 14:43:34.580979 11233 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 14:43:34.581087 11233 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 14:43:34.583351 11229 mvcc.cc:205] Tried to move back new op lower bound from 7316276078725988352 to 7316276078420529152. Current Snapshot: MvccSnapshot[applied={T|T < 7316276078495481856}]
W20260808 14:43:34.590250 11091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:34.586445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48464 (local address 127.19.144.125:34193)
0808 14:43:34.586564 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.586570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.588928 (+ 2358us) server_negotiation.cc:407] Connection header received
0808 14:43:34.589152 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.589156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.589212 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.589278 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.590093 (+ 815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48464: BlockingRecv error: recv got EOF from 127.0.0.1:48464 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 14:43:34.591820 11009 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:34.587534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43498 (local address 127.19.144.126:34025)
0808 14:43:34.587658 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.587663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.590805 (+ 3142us) server_negotiation.cc:407] Connection header received
0808 14:43:34.590996 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.591000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.591057 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.591111 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.591680 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43498: BlockingRecv error: recv got EOF from 127.0.0.1:43498 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 14:43:34.593426 11168 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:34.588268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58136 (local address 127.19.144.124:40855)
0808 14:43:34.588403 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.588408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.592360 (+ 3952us) server_negotiation.cc:407] Connection header received
0808 14:43:34.592545 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.592548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.592613 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.592677 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.593285 (+ 608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58136: BlockingRecv error: recv got EOF from 127.0.0.1:58136 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 14:43:34.598505 11091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:34.596717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48468 (local address 127.19.144.125:34193)
0808 14:43:34.596831 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.596835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.597430 (+ 595us) server_negotiation.cc:407] Connection header received
0808 14:43:34.597690 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.597694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.597780 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.597838 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.598394 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48468: BlockingRecv error: recv got EOF from 127.0.0.1:48468 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 14:43:34.606710 11091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:34.604743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48482 (local address 127.19.144.125:34193)
0808 14:43:34.604876 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.604881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.605565 (+ 684us) server_negotiation.cc:407] Connection header received
0808 14:43:34.605865 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.605869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.605930 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.605986 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.606588 (+ 602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48482: BlockingRecv error: recv got EOF from 127.0.0.1:48482 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 14:43:34.612849 11091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:34.611109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48496 (local address 127.19.144.125:34193)
0808 14:43:34.611225 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.611230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.611824 (+ 594us) server_negotiation.cc:407] Connection header received
0808 14:43:34.612040 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.612043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.612100 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.612155 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.612725 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48496: BlockingRecv error: recv got EOF from 127.0.0.1:48496 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 14:43:34.621069 11091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:34.619250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48510 (local address 127.19.144.125:34193)
0808 14:43:34.619395 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.619400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.619994 (+ 594us) server_negotiation.cc:407] Connection header received
0808 14:43:34.620205 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.620209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.620275 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.620337 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.620916 (+ 579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48510: BlockingRecv error: recv got EOF from 127.0.0.1:48510 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 14:43:34.633598 11091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:34.631574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48514 (local address 127.19.144.125:34193)
0808 14:43:34.631697 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.631702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.632395 (+ 693us) server_negotiation.cc:407] Connection header received
0808 14:43:34.632653 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.632657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.632714 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.632768 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.633469 (+ 701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48514: BlockingRecv error: recv got EOF from 127.0.0.1:48514 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 14:43:34.654424 11091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:34.652341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48526 (local address 127.19.144.125:34193)
0808 14:43:34.652477 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.652482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.653313 (+ 831us) server_negotiation.cc:407] Connection header received
0808 14:43:34.653536 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.653540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.653597 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.653690 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.654283 (+ 593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48526: BlockingRecv error: recv got EOF from 127.0.0.1:48526 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 14:43:34.691144 11091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:34.689144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48536 (local address 127.19.144.125:34193)
0808 14:43:34.689281 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.689300 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.690022 (+ 722us) server_negotiation.cc:407] Connection header received
0808 14:43:34.690261 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.690265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.690333 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.690394 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.691021 (+ 627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48536: BlockingRecv error: recv got EOF from 127.0.0.1:48536 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 14:43:34.759311 11091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:34.757712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48542 (local address 127.19.144.125:34193)
0808 14:43:34.757943 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.757952 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.758261 (+ 309us) server_negotiation.cc:407] Connection header received
0808 14:43:34.758465 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.758469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.758536 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.758638 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.759170 (+ 532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48542: BlockingRecv error: recv got EOF from 127.0.0.1:48542 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 14:43:34.891184 11091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:34.889781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48558 (local address 127.19.144.125:34193)
0808 14:43:34.889999 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:34.890005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:34.890166 (+ 161us) server_negotiation.cc:407] Connection header received
0808 14:43:34.890367 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:34.890375 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:34.890429 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:34.890523 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:34.891037 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48558: BlockingRecv error: recv got EOF from 127.0.0.1:48558 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
W20260808 14:43:35.151108 11091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 14:43:35.150023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48574 (local address 127.19.144.125:34193)
0808 14:43:35.150164 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:35.150168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:35.150208 (+ 40us) server_negotiation.cc:407] Connection header received
0808 14:43:35.150264 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:35.150266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:35.150319 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:35.150383 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 14:43:35.150979 (+ 596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48574: BlockingRecv error: recv got EOF from 127.0.0.1:48574 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 14:43:35.174717 11203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30aa7112c8af455dbe6f8d2f0ae68550" member_type: VOTER last_known_addr { host: "127.19.144.126" port: 34025 } } peers { permanent_uuid: "7118d96501eb4c42be2cbd75936907db" member_type: VOTER last_known_addr { host: "127.19.144.125" port: 34193 } } peers { permanent_uuid: "fbb26e1addae462dbe14c8d36afa1a74" member_type: VOTER last_known_addr { host: "127.19.144.124" port: 40855 } } }
I20260808 14:43:35.174944 11203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30aa7112c8af455dbe6f8d2f0ae68550 [sys.catalog]: This master's current role is: LEADER
I20260808 14:43:35.473867 11202 catalog_manager.cc:1259] Loaded cluster ID: 9c8fe5ed4e3f4c4ab0c5ceabdab0f06a
I20260808 14:43:35.473965 11202 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74: loading cluster ID for follower catalog manager: success
I20260808 14:43:35.475780 11202 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74: acquiring CA information for follower catalog manager: success
I20260808 14:43:35.477072 11202 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fbb26e1addae462dbe14c8d36afa1a74: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 14:43:35.582660 11233 catalog_manager.cc:1259] Loaded cluster ID: 9c8fe5ed4e3f4c4ab0c5ceabdab0f06a
I20260808 14:43:35.582763 11233 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db: loading cluster ID for follower catalog manager: success
I20260808 14:43:35.584525 11233 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db: acquiring CA information for follower catalog manager: success
I20260808 14:43:35.585850 11233 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7118d96501eb4c42be2cbd75936907db: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 14:43:35.669589 11304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:35.665352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39564 (local address 127.19.144.125:34193)
0808 14:43:35.665663 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:35.665671 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:35.665696 (+ 25us) server_negotiation.cc:407] Connection header received
0808 14:43:35.665820 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:35.665823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:35.665961 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:35.666063 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:43:35.666351 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:35.667002 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:35.667976 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:35.668214 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:35.668330 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:35.668680 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:43:35.669150 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:43:35.669296 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:35.669359 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 14:43:35.669420 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":79,"threads_started":1}
I20260808 14:43:35.676605 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:35.672826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41120 (local address 127.19.144.124:40855)
0808 14:43:35.673096 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:35.673100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:35.673113 (+ 13us) server_negotiation.cc:407] Connection header received
0808 14:43:35.673193 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:35.673195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:35.673309 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:35.673422 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:43:35.673598 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:35.674181 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:35.675013 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:35.675268 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:35.675385 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:35.675713 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:43:35.676168 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:43:35.676309 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:35.676374 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 14:43:35.676435 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":124,"threads_started":1}
I20260808 14:43:35.683907 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:35.680407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41134 (local address 127.19.144.124:40855)
0808 14:43:35.680534 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:35.680538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:35.680550 (+ 12us) server_negotiation.cc:407] Connection header received
0808 14:43:35.680594 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:35.680597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:35.680651 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:35.680719 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:43:35.681102 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:35.681869 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:35.682746 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:35.682987 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:35.683099 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:35.683376 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:43:35.683460 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:43:35.683544 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:35.683668 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 14:43:35.683722 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 14:43:35.684828 11304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:35.679729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39578 (local address 127.19.144.125:34193)
0808 14:43:35.679867 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:35.679874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:35.680099 (+ 225us) server_negotiation.cc:407] Connection header received
0808 14:43:35.680405 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:35.680409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:35.680472 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:35.680524 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:43:35.680922 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:35.681795 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:35.683129 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:35.683476 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:35.683563 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:35.684240 (+ 677us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:43:35.684334 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:43:35.684429 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:35.684611 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 14:43:35.684675 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 14:43:35.685989 11317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 14:43:35.679838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35272 (local address 127.19.144.126:34025)
0808 14:43:35.680125 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 14:43:35.680129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 14:43:35.680145 (+ 16us) server_negotiation.cc:407] Connection header received
0808 14:43:35.680366 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 14:43:35.680371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 14:43:35.680556 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 14:43:35.680708 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 14:43:35.681029 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:35.681591 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:35.683130 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 14:43:35.683530 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 14:43:35.683858 (+ 328us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 14:43:35.684498 (+ 640us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 14:43:35.685179 (+ 681us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 14:43:35.685514 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0808 14:43:35.685791 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 14:43:35.685864 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":106,"threads_started":1}
I20260808 14:43:35.697870 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 10934
I20260808 14:43:35.706904 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 11012
I20260808 14:43:35.715161 20033 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCQkDbp/build/debug/bin/kudu with pid 11092
2026-08-08T14:43:35Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3260 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6578 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (293359 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST